Python training UGA 2017¶

A training to acquire strong basis in Python to use it efficiently

Pierre Augier (LEGI), Cyrille Bonamy (LEGI), Eric Maldonado (Irstea), Franck Thollard (ISTERRE), Christophe Picard (LJK), Loïc Huder (ISTerre)

Object-oriented programming: encapsulation¶

Python is also an object-oriented language. Object-oriented programming is very useful and used in many libraries so it is very useful to understand how the simple object-oriented mechanisms work in Python.

For some problems, Object-oriented programming is a very efficient paradigm. Many libraries use it so it is worth understanding what is object oriented programming (POO) and when it is useful.

Concepts¶

Object¶

An object is an entity that has a state and a behavior. Objects are the basic elements of object-oriented system.

Class¶

Classes are "families" of objects. A class is a pattern that describes how objects will be built.

POO motivation: data encapsulation¶

Example: the weather stations

Let us suppose we have a set of weather stations that do measurements of wind speed and temperature. Suppose now one wants to compute some statistics on these data. A basic representation of a station will be an array of arrays: wind values and temperature values.

In [1]:
paris = [[10, 0, 20, 30, 20, 0], [1, 5, 1, -1, -1, 3]]

# get wind when temperature is maximal
idx_max_temp = paris[1].index(max(paris[1]))
print(f"max temp is of {paris[1][idx_max_temp]}°C at index {idx_max_temp} ")
print(f"wind speed at max temp = {paris[0][idx_max_temp]} km/h")

max temp is of 5°C at index 1
wind speed at max temp = 0 km/h


Many problems:

• if the number of measurements increases (e.g. having rainfall, humidity, ...) the previous indexing will not be valid (what will paris[5] will represent? wind, temperature, ..., ?)
• Code analysis is not (that) straightforward

A possible solution: create a box

We can use a dictionnary:

In [2]:
paris = {"wind": [10, 0, 20, 30, 20, 0], "temperature": [1, 5, 1, -1, -1, 3]}

# get wind when temperature is minimal
paris_temp = paris["temperature"]
idx_max_temp = paris_temp.index(max(paris_temp))

print(f"max temp is {paris_temp[idx_max_temp]}°C at index {idx_max_temp}")
print(f"wind speed at max temp = {paris['wind'][idx_max_temp]} km/h")

max temp is 5°C at index 1
wind speed at max temp = 0 km/h


• Pro

• Less error prone code (i.e. using words as keys allow to not use index numbers that are easily mistaken and lead to code that is hard to read and debug)
• Con

• The code to compute the final result is not very readable

Improvement

In [3]:
paris = {"wind": [10, 0, 20, 30, 20, 0], "temperature": [1, 5, 1, -1, -1, 3]}

def max_temp(station):
""" returns the maximum temperature available in the station"""
return max(station["temperature"])

def arg_max_temp(station):
""" returns the index of maximum temperature available in the station"""
max_temperature = max_temp(station)
return station["temperature"].index(max_temperature)

idx_max_temp = arg_max_temp(paris)

print(f"max temp is {max_temp(paris)}°C at index {arg_max_temp(paris)}")
print(f"wind speed at max temp = {paris['wind'][idx_max_temp]} km/h")

max temp is 5°C at index 1
wind speed at max temp = 0 km/h

In [4]:
# testing
import unittest

class TestWeatherStationDict0(unittest.TestCase):
"""Test the weather station """

def setUp(self):
"""Generates a well structured station (paris)"""
self.paris = {
"wind": [10, 0, 20, 30, 20, 0],
"temperature": [1, 5, 1, -1, -1, 3],
}

def test_max_temp(self):
""" test max_temp computes correctly"""
self.assertEqual(5, max_temp(self.paris))

def test_arg_max_temp(self):
""" test arg_max_temp computes correctly"""
self.assertEqual(1, arg_max_temp(self.paris))

_res = unittest.TextTestRunner(verbosity=2).run(suite)

test_arg_max_temp (__main__.TestWeatherStationDict0)
test arg_max_temp computes correctly ... ok
test_max_temp (__main__.TestWeatherStationDict0)
test max_temp computes correctly ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.002s

OK


• Pro:
• testing functions can be done separately from the rest of the code
• Con
• We rely on the fact that the dictionnaries have been built correctly (for example wind and temperature arrays have the same length).

Improvement

Define a function that builds the station (delegate the generation of the station dictionnary to a function)

In [5]:
def build_station(wind, temp):
""" Build a station given wind and temp
:param wind: (list) floats of winds
:param temp: (list) float of temperatures
"""
if len(wind) != len(temp):
raise ValueError("wind and temperature should have the same size")
return {"wind": list(wind), "temperature": list(temp)}

def max_temp(station):
""" returns the maximum temperature available in the station"""
return max(station["temperature"])

def arg_max_temp(station):
""" returns the index of maximum temperature available in the station"""
max_temperature = max_temp(station)
return station["temperature"].index(max_temperature)

paris = build_station([10, 0, 20, 30, 20, 0], [1, 5, 1, -1, -1, 3])
idx_max_temp = arg_max_temp(paris)

print(f"max temp is {max_temp(paris)}°C at index {arg_max_temp(paris)}")
print(f"wind speed at max temp = {paris['wind'][idx_max_temp]} km/h")

max temp is 5°C at index 1
wind speed at max temp = 0 km/h

In [6]:
# testing

class TestWeatherStationDict1(unittest.TestCase):
"""Test the weather station """

def setUp(self):
"""Generates a well structured station (paris)"""
self.paris = build_station([10, 0, 20, 30, 20, 0], [1, 5, 1, -1, -1, 3])

def test_build_station_with_iterable(self):
""" Tests that the station can be generated from iterables """
station = build_station(range(10), range(10))
self.assertEqual(station["wind"][0], 0)
self.assertEqual(station["wind"][-1], 9)
self.assertEqual(station["temperature"][0], 0)
self.assertEqual(station["temperature"][-1], 9)

def test_wrong_build(self):
""" Tests that the station generation throws an error
if wind and temperature do not have the same size"""
with self.assertRaises(ValueError):

def test_max_temp(self):
""" test max_temp computes conrrectly"""
self.assertEqual(5, max_temp(self.paris))

def test_arg_max_temp(self):
""" test arg_max_temp computs correctly"""
self.assertEqual(1, arg_max_temp(self.paris))

_res = unittest.TextTestRunner(verbosity=2).run(suite)

test_arg_max_temp (__main__.TestWeatherStationDict1)
test arg_max_temp computs correctly ... ok
test_build_station_with_iterable (__main__.TestWeatherStationDict1)
Tests that the station can be generated from iterables ... ok
test_max_temp (__main__.TestWeatherStationDict1)
test max_temp computes conrrectly ... ok
test_wrong_build (__main__.TestWeatherStationDict1)
Tests that the station generation throws an error ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.009s

OK


• If the dedicated function build_station is used, the returned dictionary is well structured.
• If one changes build_station, only max_temp and arg_max_temp have to be changed accordingly

Object oriented in a nutshell¶

A class defines a template used for building object. In our example, the class (named WeatherStation) defines the specifications of what is a weather station (i.e, a weather station should contain an array for wind speeds, named "wind", and an array for temperatures, named "temp"). paris should now be an object that answers to these specifications. Is is called an instance of the class WeatherStation.

When defining the class, we need to define how to build it (special "function" __init__).

In [7]:
class WeatherStation(object):
""" A weather station that holds wind and temperature

:param wind: any ordered iterable
:param temperature: any ordered iterable

wind and temperature must have the same length.

"""

def __init__(self, wind, temperature):
self.wind = list(wind)
self.temp = list(temperature)
if len(self.wind) != len(self.temp):
raise ValueError(
"wind and temperature should have the same size"
)

def max_temp(self):
""" returns the maximum temperature recorded in the station"""
return max(self.temp)

def arg_max_temp(self):
""" returns the index of (one of the) maximum temperature recorded in the station"""
return self.temp.index(self.max_temp())

paris = WeatherStation([10, 0, 20, 30, 20, 0], [1, 5, 1, -1, -1, 3])
# OR paris = WeatherStation(wind=[10, 0, 20, 30, 20, 0], temperature=[1, 5, 1, -1, -1, 3])
idx_max_temp = paris.arg_max_temp()

print(f"max temp is {paris.max_temp()}°C at index {paris.arg_max_temp()}")
print(f"wind speed at max temp = {paris.wind[idx_max_temp]} km/h")

max temp is 5°C at index 1
wind speed at max temp = 0 km/h


The max_temp and the arg_max_temp are now part of the class WeatherStation. Functions attached to classes are named methods. Similary, wind and temp lists are also now part this class. Variables attached to classes are named members or attributes.

An object (here paris) thus contains both attributes (holding data for example) and methods to access and/or process the data.

Testing

In [8]:
class TestWeatherStation(unittest.TestCase):
"""Test the weather station """

def setUp(self):
"""Generates a well structured station (paris)"""
self.paris = WeatherStation(
[10, 0, 20, 30, 20, 0],
[1, 5, 1, -1, -1, 3],
)

def test_building_with_good_input_arrays(self):
""" test that things goes smoothly if the input are correct"""
self.assertEqual(0, self.paris.wind[1])
self.assertEqual(5, self.paris.temp[1])

def test_building_with_input_iterables(self):
""" test that things goes smoothly if the input are correct"""
r_station = WeatherStation(range(10), range(10))
self.assertEqual(4, r_station.wind[4])
self.assertEqual(5, r_station.temp[5])

""" test that an exception is raised with incorrect inputs"""
with self.assertRaises(ValueError):
bad_station = WeatherStation([10, 0, 20, 30, 20, 0], [1, 5, 1])

def test_max_temp(self):
""" test test_max_temp function"""
self.assertEqual(5, self.paris.max_temp())

def test_arg_max_temp(self):
""" test arg_max_temp function"""
self.assertEqual(1, self.paris.arg_max_temp())

_res = unittest.TextTestRunner(verbosity=2).run(suite)

test_arg_max_temp (__main__.TestWeatherStation)
test arg_max_temp function ... ok
test that an exception is raised with incorrect inputs ... ok
test_building_with_good_input_arrays (__main__.TestWeatherStation)
test that things goes smoothly if the input are correct ... ok
test_building_with_input_iterables (__main__.TestWeatherStation)
test that things goes smoothly if the input are correct ... ok
test_max_temp (__main__.TestWeatherStation)
test test_max_temp function ... ok

----------------------------------------------------------------------
Ran 5 tests in 0.009s

OK


• Add a method (perceived_temp) that takes as input a temperature and wind and return the perceived temperature, i.e. taking into account the wind chill effect.
• Modify max_temp and arg_max_temp so that they take an additional optional boolean parameter (e.g. perceived default to False). If perceived is False, the methods have the same behaviour as before. If perceived is True, the temperatures to process are the perceived temperatures.
• Code so that all the tests are passing.
• Add a tests for a method (perceived_temperatures) that returns an array containing all the perceived temperatures.
• Code the method perceived_temperatures so that all tests are passing.
In [9]:
class WeatherStation(object):
""" A weather station that holds wind and temperature """

def __init__(self, wind, temperature):
""" initialize the weather station.
Precondition: wind and temperature must have the same length
:param wind: any ordered iterable
:param temperature: any ordered iterable"""
self.wind = list(wind)
self.temp = list(temperature)
if len(self.wind) != len(self.temp):
raise ValueError(
"wind and temperature should have the same size"
)

def perceived_temp(self, index):
""" computes the perceived temp according to
https://en.wikipedia.org/wiki/Wind_chill
i.e. The standard Wind Chill formula for Environment Canada is:
apparent = 13.12 + 0.6215*air_temp - 11.37*wind_speed^0.16 + 0.3965*air_temp*wind_speed^0.16

:param index: the index for which the computation must be made
:return: the perceived temperature"""
air_temp = self.temp[index]
wind_speed = self.wind[index]
# Perceived temperature does not have a sense without wind...
if wind_speed == 0:
apparent_temp = air_temp
else:
apparent_temp = 13.12 + 0.6215*air_temp \
- 11.37*wind_speed**0.16 \
+ 0.3965*air_temp*wind_speed**0.16
# Let's round to avoid trailing decimals...
return round(apparent_temp, 2)

def perceived_temperatures(self):
""" Returns an array of percieved temp computed from the temperatures and wind speed data """
apparent_temps = []
for index in range(len(self.wind)):
# Reusing the method perceived_temp defined above
apparent_temperature = self.perceived_temp(index)
apparent_temps.append(apparent_temperature)
return apparent_temps

def max_temp(self, perceived=False):
""" returns the maximum temperature record in the station"""
if perceived:
apparent_temp = self.perceived_temperatures()
return max(apparent_temp)
else:
return max(self.temp)

def arg_max_temp(self, perceived=False):
""" returns the index of (one of the) maximum temperature record in the station"""
if perceived:
temp_array_to_search = self.perceived_temperatures()
else:
temp_array_to_search = self.temp
return temp_array_to_search.index(self.max_temp(perceived))

class TestWeatherStation(unittest.TestCase):
"""Test the weather station """

def setUp(self):
"""Generates a well formed station (paris)"""
self.paris = WeatherStation(
[10, 50, 20, 30, 20, 0],
[1, 5, 1, -1, -1, 3],
)

def test_building_with_good_input_arrays(self):
""" test things goes smoothly if input are correct"""
self.assertEqual(50, self.paris.wind[1])
self.assertEqual(5, self.paris.temp[1])

def test_building_with_input_iterables(self):
""" test things goes smoothly if input are correct"""
r_station = WeatherStation(range(10), range(10))
self.assertEqual(4, r_station.wind[4])
self.assertEqual(5, r_station.temp[5])

""" test exception is raised with bad entry"""
with self.assertRaises(ValueError):
bad_station = WeatherStation([10, 50, 20, 30, 20, 0], [1, 5, 1])

def test_perceived_with_null_wind(self):
""" test that the perceived temp is the same without wind"""
index = 5
assert self.paris.wind[index] == 0
self.assertEqual(self.paris.temp[index], self.paris.perceived_temp(index))

def test_perceived(self):
""" test that the perceived temp is lower than the regular temp with wind """
index = 2
assert self.paris.wind[index] > 0
self.assertTrue(self.paris.perceived_temp(index) < self.paris.temp[index])

def test_perceived_temperatures(self):
""" test that the perceived temperatures generation works"""
apparent_temps = self.paris.perceived_temperatures()
self.assertTrue(isinstance(apparent_temps, list))
self.assertEqual(len(apparent_temps), len(self.paris.temp))
for i in range(0, len(apparent_temps)):
self.assertLessEqual(apparent_temps[i], self.paris.temp[i])

def test_max_temp_no_perceived(self):
""" test max temp function """
self.assertEqual(5, self.paris.max_temp())

def test_arg_max_temp_no_perceived(self):
""" test arg_max_temp function"""
self.assertEqual(1, self.paris.arg_max_temp())

def test_max_temp_perceived(self):
""" test max temp function"""
self.assertEqual(3, self.paris.max_temp(perceived=True))

def test_arg_max_temp_no_perceived(self):
""" test arg_max_temp function"""
self.assertEqual(5, self.paris.arg_max_temp(perceived=True))

_res = unittest.TextTestRunner(verbosity=2).run(suite)

test_arg_max_temp_no_perceived (__main__.TestWeatherStation)
test arg_max_temp function ... ok
test exception is raised with bad entry ... ok
test_building_with_good_input_arrays (__main__.TestWeatherStation)
test things goes smoothly if input are correct ... ok
test_building_with_input_iterables (__main__.TestWeatherStation)
test things goes smoothly if input are correct ... ok
test_max_temp_no_perceived (__main__.TestWeatherStation)
test max temp function ... ok
test_max_temp_perceived (__main__.TestWeatherStation)
test max temp function ... ok
test_perceived (__main__.TestWeatherStation)
test that the perceived temp is lower than the regular temp with wind ... ok
test_perceived_temperatures (__main__.TestWeatherStation)
test that the perceived temperatures generation works ... ok
test_perceived_with_null_wind (__main__.TestWeatherStation)
test that the perceived temp is the same without wind ... ok

----------------------------------------------------------------------
Ran 9 tests in 0.014s

OK


• The wind array was changed to have different maximum temperatures for the air and perceived temperatures : for air temperatures, the max is 5°C (with a wind speed 50 km/h). For perceived temperatures, the max is 3°C (as the wind speed is 0).
• It was a choice to set the apparent/perceived temperature to the air temperature if the wind speed is 0 so the tests were written with this in mind. Testing such choices allows to have clear inputs/outputs.
• isinstance allows to test the type of an object (in this case, we test if apparent_temps is a list)
• When testing boolean in if structures: use if perceived: rather than if perceived == True:. It is equivalent but clearer and shorter !

Coming next¶

What if we now have a weather station that also measure humidity ?

Do we need to rewrite everything ?

What if we rewrite everything and we find a bug ?

Here comes inheritance