# Python training UGA 2017¶

A training to acquire strong basis in Python to use it efficiently

Pierre Augier (LEGI), Cyrille Bonamy (LEGI), Eric Maldonado (Irstea), Franck Thollard (ISTerre), Christophe Picard (LJK), Loïc Huder (ISTerre)

# Standard types and basic statements¶

## Function calls¶

There are built-in functions and the developers can of course define other functions. To call a function:

In [1]:
print("hello")

hello


Some functions return a result.

In [2]:
round(1.2)

Out[2]:
1

It's common to store the result in a variable:

In [3]:
my_var = round(1.2)


which can then be used:

In [4]:
print(my_var)

1


## Few standard types¶

• Simple types (int, float, bool, complex)
• Standard type str
• Standard type list
• Standard type tuple

### int (integers)¶

In [5]:
a = 4
c = -10

# binary notation (base 2)
b = 0b010

# octal notation (base 8)
o = 0o011

h = 0x1cf0

a = int('1')       # base 10
a = int('111', 2)  # base 2
a = int('70', 8)   # base 8
a = int('16', 16)  # base 16


Remark: int in Python 3 are impressive! No limit! See https://docs.python.org/3.1/whatsnew/3.0.html#integers

## Arithmetic operations¶

In [6]:
print(10 + 3)
print(10 - 3)
print(10 * 3)
print(10 / 3)  # float division
print(10 // 3)  # integer division
print(10 % 3)

13
7
30
3.3333333333333335
3
1


### bool (booleans)¶

In [7]:
b = bool('1')
b = False
b = True

##### Comparison operations (bool)¶
• == equal
• != différent
• < inferior
• <= inferior or equal
• > superior
• >= superior or equal
##### Keyword is: check identity¶
In [8]:
a = None
print(a is None)
print(a is not None)

True
False

##### Keywords and and or¶
In [9]:
True and True

Out[9]:
True
In [10]:
True and False

Out[10]:
False
In [11]:
False and False

Out[11]:
False
In [12]:
True or True

Out[12]:
True
In [13]:
True or False

Out[13]:
True
In [14]:
False or False

Out[14]:
False

### float (real, double precision) and complex¶

In [15]:
# float
a = float('1')
a = 1.234
a = 1e2
a = -1e-2
a = .2

In [16]:
# complex (2 floats)
c = complex('1')
c = 1 + 2j
print(c, c.real, c.imag)

(1+2j) 1.0 2.0


Remark: notation var_name.attr_name to access to an attribute of an object.

## Warning about floating-point arithmetic and numerical errors!¶

In [17]:
b = 1e16
c = 1.2 + b
d = c - b
print(d)

2.0


Very general issue (not Python):

### Standard type str¶

In [18]:
s = 'hello'
s = "hello"

s = ('How is it possible to write a very very '
'very long string with lines limited to 79 characters?')

s = """Strings on
more thand
one line.
"""
print(s)

Strings on
more thand
one line.



Warning: big difference between Python 2 and Python 3. In Python 3, str are unicode and there is another type bytes.

#### Methods of the type str¶

Objects of built-in types have methods associated with their type (object oriented programming). The built-in function dir returns a list of name of the attributes. For a string, these attributes are python system attributes (with double-underscores) and several public methods:

In [19]:
s = 'abcdef'
print(dir(s))

['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'capitalize', 'casefold', 'center', 'count', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'format_map', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']


To access an attribute of an object (here, the method str.startswith), we use the dot:

In [20]:
s.startswith('a')

Out[20]:
True

### standard type str¶

#### function str.format¶

Docstring:
S.format(*args, **kwargs) -> str

Return a formatted version of S, using substitutions from args and kwargs.
The substitutions are identified by braces ('{' and '}').

In [21]:
a = 1.23456789
'a = {}'.format(a)

Out[21]:
'a = 1.23456789'
In [22]:
'a = {:.4f}'.format(a)

Out[22]:
'a = 1.2346'
In [23]:
'a = {:8.4f}'.format(a)

Out[23]:
'a =   1.2346'
In [24]:
'a = {:.4e} (scientific notation)'.format(a)

Out[24]:
'a = 1.2346e+00 (scientific notation)'
In [25]:
print('{}\t{}\t{}'.format(1, 2, 3))

1	2	3


### standard type str¶

#### New in Python 3.6: format strings¶

In [26]:
a = 1.23456789
f'a = {a}'

Out[26]:
'a = 1.23456789'
In [27]:
f'a = {a:.4f}'

Out[27]:
'a = 1.2346'
In [28]:
f'a = {a:8.4f}'

Out[28]:
'a =   1.2346'
In [29]:
f'a = {a:.4e} (scientific notation)'

Out[29]:
'a = 1.2346e+00 (scientific notation)'
In [30]:
print(f'{1}\t{1+1}\t{2+1}')

1	2	3


### standard type str¶

Strings are immutable "sequences".

• lookup
In [31]:
s = 'abcdef'
print('a' in s)
print('hello' not in s)

True
True

• We can get an element of a string (index starts from 0):
In [32]:
print(s[0])

a

• since strings are immutable, they can not be modified inplace. If we try, we get an error:
s[0] = 'b'

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-55620f378bce> in <module>()
----> 1 s[0] = 'b'

TypeError: 'str' object does not support item assignment

• since strings are sequences, they can be "sliced" (we will soon study in details this powerful notation):
In [33]:
s[1:3]

Out[33]:
'bc'
• it is very simple to manipulate strings in many ways:
In [34]:
print((s.capitalize() + ' ' + s.upper() + '\n') * 4 )

Abcdef ABCDEF
Abcdef ABCDEF
Abcdef ABCDEF
Abcdef ABCDEF



### Slicing¶

Very general, can be used on all sequences as str, list, etc... Not simple for beginners but very powerfull (see here and here).

Python indexes and slices for a six-element str. Indexes enumerate the elements, slices enumerate the spaces between the elements.

Index from rear:    -6  -5  -4  -3  -2  -1
Index from front:    0   1   2   3   4   5
+---+---+---+---+---+---+
| a | b | c | d | e | f |
+---+---+---+---+---+---+
Slice from front:  0   1   2   3   4   5   6
Slice from rear:  -6  -5  -4  -3  -2  -1   None

In [35]:
s = 'abcdef'
# s[start:stop:step]
s[2:6:2]

Out[35]:
'ce'
In [36]:
# s[start:stop]
s[2:6]

Out[36]:
'cdef'
In [37]:
# s[start:]
s[1:]

Out[37]:
'bcdef'
In [38]:
# s[:stop]
s[:2]

Out[38]:
'ab'
In [39]:
# step = -1: goes through the string in reverse order
s[::-1]

Out[39]:
'fedcba'

### Do it yourself¶

Suppose we have a string representing a header line of the form:

In [40]:
s = ' wind;temperature;;pressure '

• Remove leading and ending blanks (see str.replace and str.strip)
• Extract the first field (e.g. using find method and slicing)
• Extract the last field (e.g. using rfind method and slicing)
• Check for empty field (e.g. find ";;" pattern)
• Remove empty field (e.g. using replace method)
• BONUS : Extract the second field (tip :find can take an argument that tells where to start the search)

#### A possible solution:¶

In [41]:
s = ' wind;temperature;;pressure '
s = s.strip()
print("--{}--".format(s))
# extract the first field
idx = s.find(";")
s0 = s[0:idx]
print(s0)
# extract the second field
idx1 = s.find(";", idx+1) # start the search after the first ";"
s1 = s[idx+1:idx1]
print(s1)
# extract the last field
idx2 = s.rfind(";")
s2 = s[idx2+1:]
print(s2)
idx_first_empty_field = s.find(";;")
print(idx_first_empty_field)
# remove empty field
s_no_empty = s.replace(";;", ";")
print(s_no_empty)

--wind;temperature;;pressure--
wind
temperature
pressure
16
wind;temperature;pressure


### standard type list¶

A list is a mutable sequence of (possibly inhomogeneous) elements.

In [42]:
type([0, 'a'])

Out[42]:
list
In [43]:
# create an empty list
l = []
# fill the list (with the function append)
l.append('2')
# fill the list (with the function extend)
l.extend([6, 3.])
print(l)

['2', 6, 3.0]

In [44]:
# concatenate lists with the operator +
print(l + ['hello', 3])

['2', 6, 3.0, 'hello', 3]

In [45]:
# get values
print(l[0], l[2], l[-2])
# slicing
print(l[0:2])

2 3.0 6
['2', 6]


### standard type tuple¶

A tuple is a immutable sequence of (possibly inhomogeneous) elements.

Remark: when you need a sequence that won't be modified, tuple is usually more efficient than list.

In [46]:
t = 0, 'a', 1.2
t1 = (5, 'hello')
t2 = tuple([1.1, 2])
type(t)

Out[46]:
tuple
In [47]:
t[1]  # indexing

Out[47]:
'a'
In [48]:
t[1:]  # slicing

Out[48]:
('a', 1.2)
In [49]:
a, b = t1  # tuple assigment
print(b)

hello


## Mutable and immutable objects¶

The objects of type str, int, float, bool are immutable. They can not be modified. Of course, a name that points towards an integer can point towards a different integer.

In [50]:
i = 1
i = i + 2  # (or i += 2)
print(i)
i = 10
print(i)

3
10


Here, the objects 1 and 3 have not been modified.

## Mutable and immutable objects¶

An object of type list is mutable:

In [51]:
l = [0, 5]
print(l)
l.append('hello')
print(l)

[0, 5]
[0, 5, 'hello']


Here, the object list tagged by the name l has been modified inplace.

## References and del keyword¶

del removes a reference. If an object in not binded to any names, Python can delete it from its internal memory.

In [52]:
l = ['a', 'b']
del l[1]
print(l)

['a']


### More on slicing¶

Very general, can be used on all sequences as str, list, etc... Not simple for beginners but very powerfull (see here and here).

Python indexes and slices for a six-element str. Indexes enumerate the elements, slices enumerate the spaces between the elements.

Index from rear:    -6  -5  -4  -3  -2  -1
Index from front:    0   1   2   3   4   5
+---+---+---+---+---+---+
| a | b | c | d | e | f |
+---+---+---+---+---+---+
Slice from front:  0   1   2   3   4   5   6
Slice from rear:  -6  -5  -4  -3  -2  -1   0

In [53]:
s = 'abcdef'
# s[start:stop:step]
s[2:6:2]

Out[53]:
'ce'

### More on slicing¶

#### Assigment to mutable object¶

In [54]:
l = [0, 1, 2, 3, 4, 5]
l1 = l  # assigment to a new name l1 (no copy of the object).
# the names l and l1 points towards the same object.
l1.append('a')
print(l1)
print(l)

[0, 1, 2, 3, 4, 5, 'a']
[0, 1, 2, 3, 4, 5, 'a']


#### Shallow copy¶

In [55]:
l = [0, 1, 2, 3, 4, 5]
l1 = l[:] # shallow copy of l
l1.append('a')
print(l1)
print(l)

[0, 1, 2, 3, 4, 5, 'a']
[0, 1, 2, 3, 4, 5]


### More on slicing¶

Other examples of slices for a six-element list. Indexes enumerate the elements, slices enumerate the spaces between the elements.

In [56]:
a = [0, 1, 2, 3, 4, 5]
all([
len(a) == 6,
a[1:] == [1, 2, 3, 4, 5],
a[:5] == [0, 1, 2, 3, 4],
a[0] == 0,
a[:-2] == [0, 1, 2, 3],
a[5] == 5,
a[1:2] == [1],
a[-1] == 5,
a[1:-1] == [1, 2, 3, 4],
a[-2] == 4,
])

Out[56]:
True

### Do it yourself¶

Suppose we have the string containing header line.

In [57]:
s = 'wind;temperature;pressure'

• Extract the list of items (i.e. "wind", "temperature", "pressure"; see str.split).
• Add "Snow level" to the list of items (e.g. using append)
• Build a new header such that items are capitalized (e.g. using the methods str.join and str.capitalize and iterating on the list)

#### A possible solution:¶

In [58]:
s = 'wind;temperature;pressure'
list_items = s.split(";")
print(list_items)
list_items.append("snow level".capitalize())
list_items[0] = list_items[0].capitalize()
list_items[1] = list_items[1].capitalize()
list_items[2] = list_items[2].capitalize()

print(list_items)
";".join(list_items)

['wind', 'temperature', 'pressure']
['Wind', 'Temperature', 'Pressure', 'Snow level']

Out[58]:
'Wind;Temperature;Pressure;Snow level'

### The function range¶

The function returns a range object:

In [59]:
# start, stop, step
range(1, 8, 2)

Out[59]:
range(1, 8, 2)

We can make a list with the range object:

In [60]:
# start, stop, step
list(range(1, 8, 2))

Out[60]:
[1, 3, 5, 7]
In [61]:
# start, stop (step=1)
list(range(2, 8))

Out[61]:
[2, 3, 4, 5, 6, 7]
In [62]:
# stop argument (start=0, step=1)
list(range(8))

Out[62]:
[0, 1, 2, 3, 4, 5, 6, 7]

### Do it yourself¶

Build a list of odd numbers in decreasing order.

## Conditions: if, elif, else¶

if expression:
statement(s)
else:
statement(s)


The statement contains the block of code that executes if the conditional expression in the if statement resolves to 1 or a TRUE value.

In [63]:
a = 0
if a == 0:
print('a is equal to 0.')

a is equal to 0.

In [64]:
a = 1
if a < 0:
print('a is negative.')
elif a == 0:
print('a is equal to 0.')
elif a > 0:
print('a is positive.')
else:
print("I don't know.")

a is positive.


## Loops¶

### Loops with the keyword while¶

In [65]:
i = 0
while i < 4:
i += 1
print('i =', i)

i = 4

In [66]:
i = 0
while i < 4:
i += 1
print('i =', i)


i = 1
i = 2
i = 3
i = 4


### Do it yourself¶

• Edit a script with the spyder editor that calculates the average of a set of numbers. For example numbers = [67, 12, 2, 9, 23, 5]

• using the functions sum and len
• manually, using the keyword while
• Run the script

• in spyder,
• in a ipython session opened from another terminal,
• with the command python.

#### A possible solution:¶

In [67]:
numbers = [67, 12, 2, 9, 23, 5]
local_sum = 0
i = 0
while i < len(numbers):
local_sum = local_sum + numbers[i]
i = i+1
avg = local_sum / len(numbers)
print(avg)

19.666666666666668


### Loops with the keyword for¶

In [68]:
for index in range(5):
print(index, end=', ')

0, 1, 2, 3, 4,
In [69]:
cities = ["grenoble", "paris", "berlin", "london"]
for city in cities:
print(city, end=", ")

grenoble, paris, berlin, london,
In [70]:
# the built-in function enumerate is very useful
for index, city in enumerate(cities):
print('({}, {})'.format(index, city))
cities[index] = city.capitalize()

(0, grenoble)
(1, paris)
(2, berlin)
(3, london)

In [71]:
cities

Out[71]:
['Grenoble', 'Paris', 'Berlin', 'London']

### Loops: keywords continue and break¶

• continue: passes the block in the loop and continues the loop.
In [72]:
for x in range(1, 8):
if x == 5:
continue
print(x, end=', ')

1, 2, 3, 4, 6, 7,
• break: stop the loop.
In [73]:
for x in range(1, 11):
if x == 5:
break
print(x, end=', ')

1, 2, 3, 4,

### Do it yourself¶

• Simplify your script by using a for loops.

• In ipython, try to understand how the function enumerate works. Use it in your script.

#### A possible solution:¶

In [74]:
l = [67, 12, 2, 9, 23, 5]
local_sum = 0
for e in l:
local_sum += e
avg = local_sum / len(l)
print(avg)

19.666666666666668


### Do it yourself¶

We build a list:

In [75]:
from random import shuffle, randint

n = 20
i = randint(0, n-1)
print('integer remove from the list:', i)
l = list(range(n))
l.remove(i)
shuffle(l)
print('shuffled list: ', l)

integer remove from the list: 19
shuffled list:  [9, 17, 15, 14, 1, 3, 18, 7, 6, 16, 4, 0, 12, 5, 13, 2, 11, 8, 10]


One element has been removed:

• Find this element (given that you can change the ordering of l).
• Find this element (given that you cannot change the ordering of l).

#### A possible solution:¶

In [76]:
# we can change ordering, let's sort
print(l)
l_sorted = sorted(l)
print(l_sorted)
found = None
for idx, elem in enumerate(l_sorted):
if elem != idx:
found = idx
break
if found is None:
found = len(l)
print("missing ", idx)


[9, 17, 15, 14, 1, 3, 18, 7, 6, 16, 4, 0, 12, 5, 13, 2, 11, 8, 10]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
missing  18

In [77]:
# we cannot sort -> higher complexity
for elem in range(len(l)+1):
if elem not in l:
break
print("missing ", elem)

missing  19

In [78]:
# another solution
actual_sum = sum(l)
len_l = len(l)
original_sum = (len_l + 1) * (len_l) // 2
print("missing ", original_sum - actual_sum)

missing  19


## Exceptions and try, except syntax¶

Exceptions and errors are common in all codes. There is a good system to handle them in Python. Let's first see what gives such buggy code

l = ['a']
i = 1
print(l[i])


When these lines are executed, Python stops its execution and print a traceback:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-30-8df9cec1a0ec> in <module>()
1 l = ['a']
2 i = 1
----> 3 print(l[i])

IndexError: list index out of range


## Exceptions and try, except syntax¶

Handling exception:

In [79]:
l = ['a']
i = 1
try:
print(l[i])
except IndexError as e:
print(e)

list index out of range


Remark: never use

except:


It means "except all errors and exceptions". A user Control-C is an exception (KeyboardInterrupt) so it would be caught and have no effect.

## Full syntax¶

try:
...
except <exception1> as e1:
...
except <exception2> as e2:
...
else:
...
finally:
...

• ArithmeticError
• ZeroDivisionError
• IndexError
• KeyError
• AttributeError
• IOError
• ImportError
• NameError
• SyntaxError
• TypeError

### Do it yourself:¶

For each line of this string, append a variable of correct type in a list (i.e. "hello" should stay hello, 2 should become an int and 1.5 a float). Do it by catching errors.

Hints:

• int("a") and float("a") raise ValueError
• the above str_variable can be split using "\n"
In [80]:
str_variables = """hello
1.5
2"""

the_list_you_should_get = ["hello", 1.5, 2]


#### A possible solution:¶

In [81]:
split_list = []
for value in str_variables.split("\n"):
try:
value = int(value)
except ValueError:
print(value, "is not a int")
try:
value = float(value)
except ValueError:
print(value, 'is not a float')
split_list.append(value)

print(split_list)

hello is not a int
hello is not a float
1.5 is not a int
['hello', 1.5, 2]


### Bonus: the Easier to Ask for Forgiveness than Permission (EAFP) principle¶

There are several ways to perform the previous task:

• In the solution presented above, we try first the conversion of the string in number and ask for forgiveness if we encounter an error (by using the except).
• Or we can test if the string is a number, using for example str.isdigit() or str.isdecimal() (asking for permission) and do the conversion only in this case.

Both approches will lead to the same result so it is rather a question of principle. In Python, it is generally prefered to use try and except to follow the EAFP principle. In this particular case, using try and except is far easier !

In [82]:
split_list = []
for value in str_variables.split("\n"):
if value.isdecimal():
value = int(value)
else:
print(value, "is not a int")
int_part, sep, dec_part = value.partition('.')
if int_part.isdecimal() and dec_part.isdecimal():
value = float(value)
else:
print(value, 'is not a float')
split_list.append(value)

print(split_list)

hello is not a int
hello is not a float
1.5 is not a int
['hello', 1.5, 2]

In [ ]: