{"id":524,"date":"2021-01-04T14:37:35","date_gmt":"2021-01-04T14:37:35","guid":{"rendered":"https:\/\/enter77.ius.edu\/cjkimmer\/?page_id=524"},"modified":"2021-01-04T14:57:56","modified_gmt":"2021-01-04T14:57:56","slug":"python-collections","status":"publish","type":"page","link":"https:\/\/enter77.ius.edu\/cjkimmer\/python-collections\/","title":{"rendered":"Python Collections"},"content":{"rendered":"\n

Python has a variety of built-in data structures that we will use in this course\u2014lists, tuples, sets, and dictionaries. All of these are similar in that they represent collections of objects, but they each have their own strengths and weaknesses relative to each other for different tasks. They are also all objects, and so they have their own member functions that we may use to access or manipulate the data they contain. All four of the data structures mentioned above are heterogeneous<\/strong> which means they can hold a collection of different data types. This in contrast to a Java array, for instance, which must declare the type it can hold. For instance a Java array of integers can only hold integer values while a Python list could hold an integer, a floating point number, and a string or any other object we would care to place in the collection.<\/p>\n\n\n\n

A good reference on all the information for each collection below is available in the official Python documentation here<\/a>.<\/p>\n\n\n\n

Lists<\/h2>\n\n\n\n

A list in Python can be created in multiple ways. Two ways allow us to create an empty list to which we can add items later. A third way allows us to create a list that is initialized to hold prescribed values. This code snippet illustrates all three ways:<\/p>\n\n\n\n

empty_list = list() # we call the list() constructor here\nanother_empty_list = [] # this is a more typical way to create an empty list \nlist_with_values = [1, 2.3, \"informatics\"] # list is heterogenous. Comma-separated list of values to initialize<\/pre>\n\n\n\n
The initialization of another empty list above is said to be more pythonic<\/strong> since it\u2019s the typical way to program such a statement in Python. The previous line initializing empty_list<\/code> would be less typical in a program, but it does illustrate to us that list is a class while the objects are instances of that class. The list<\/code> class is part of Python, so it is a built-in class. Since it\u2019s a class, it has member functions that we can use. One of the most common is list.append()<\/code> which lets us add to the end of the list:<\/p>\n\n\n\n
another_empty_list = [] # this is a more typical way to create an empty list \nlist_with_values = [1, 2.3, \"informatics\"] # list is heterogenous. Comma-separated list of values to initialize\n\nanother_empty_list.append(1) # another_empty_list now equals [1]\nlist_with_values.append(\"i427\") # now holds [1,2.3,\"informatics\",\"i427\"]<\/pre>\n\n\n\n
There is also a list.extend()<\/code> method which lets us append multiple values from one list onto another<\/p>\n\n\n\n
another_empty_list = [] # this is a more typical way to create an empty list \nlist_with_values = [1, 2.3, \"informatics\"] # list is heterogenous. Comma-separated list of values to initialize\nanother_empty_list.append(1) # another_empty_list now equals [1] list_with_values.append(\"i427\") # now holds [1,2.3,\"informatics\",\"i427\"] \nanother_empty_list.extend(list_with_values) # now holds [1,1,2.3,\"informatics\",\"i427\"] \n\nanother_empty_list + list_with_values # + operator acts just like extend, now holds [1,1,2.3,\"informatics\",\"i427\"]<\/pre>\n\n\n\n
Finally, the code below illustrates a key property of a list\u2014random access by index. Python lists act like Java arrays in that each item has a position in the sequence. The first item is at index 0, the second at index 1, and so on.<\/p>\n\n\n\n
>>> list_with_values = [1, 2.3, \"informatics\"] # list is heterogenous. Comma-separated \udbff\udc0b\u2192 list of values to initialize\n>>> print(list_with_values[0])\n1\n>>> print(list_with_values[1])\n2.3\n>>> print(list_with_values[2])\ninformatics\n>>> print(list_with_values[-1])\ninformatics\n>>> print(list_with_values[-2])\n2.3\n>>> print(len(list_with_values))\n3<\/pre>\n\n\n\n
The previous example illustrates that we can access using negative indices: -1 is the last item, -2 the next to last, and so on. Even though list<\/code> is a class, we determine the length of a list<\/code> using the built-in len()<\/code> function and supplying the list<\/code> as an argument. We do not use a member function in the list class to determine the length. Access by index is one of the key traits of a list, so whenever we are holding a collection of data and we care that an item is first, second, third … we would choose list<\/code> as our container.<\/p>\n\n\n\n
Traversing a list<\/h3>\n\n\n\n
It is common in computing to need to do something to every item in a list<\/code>. To accomplish these repeated (once per element) operations, we typically use iteration–a for loop– in programming. Iterating over these collections in Python is similar to using enhanced for loops in Java. A dummy variable<\/strong> acts as a placeholder for each value in the collection in turn. We are free to choose its name. The dummy variable is not an index; it is a value in the collection. Short variable names like i<\/code> are appropriate for an index but not a value. item<\/code> or value<\/code> are appropriate generic names, or a variable name that describes the contents o the collection would be appropriate. The only times single letter variable names would be appropriate would be when implementing mathematical operations or formula, like adding up elements, on a list. Then names like i,j<\/code>, or k<\/code> would be appropriate for int<\/code> values or x, y<\/code>, or z<\/code> may be appropriate for float<\/code>.<\/p>\n\n\n\n
>>> list_of_names = [\"Larry\", \"Moe\", \"Curly\"]\n>>> for item in list_of_names: # generic dummy variables ... print(item)\n...\nLarry\nMoe\nCurly\n>>> for name in list_of_names: # more meanginful dummy name ... print(name)\n...\nLarry\nMoe\nCurly<\/pre>\n\n\n\n
Python provides the enumerate() <\/code>method for cases when we need the value in the list as well as the index of that value. In this case, we get to name two<\/em> dummy variables. The first is the index, and i is an appropriate dummy variable name in this case. The second is the value in the list, and we should name it as described above.<\/p>\n\n\n\n
>>> list_of_names = [\"Larry\",\"Moe\",\"Curly\"]\n>>> for i,name in enumerate(list_of_names):\n...     print(i)\n...     print(name)\n...\n0\nLarry\n1\nMoe\n2\nCurly<\/pre>\n\n\n\n
What is happening in the previous example is that the enumerate<\/code> method is yielding (i.e. returning each iteration) a tuple<\/strong> of 2 values which are simultaneously assigned to i<\/code>, name<\/code> for each iteration of the loop. Also note that since i<\/code> is an index, it’s an appropriate variable name in this case. We don\u2019t know what a tuple is yet, so let\u2019s find out. <\/p>\n\n\n\n
Sets and Tuples<\/h2>\n\n\n\n
Aset<\/code> and a tuple<\/code> are similar to a list in many ways, but each also has at least one key difference. A set<\/code> represents a mathematical set, so it cannot hold duplicate values and has no particular ordering. This is in contrast to a list<\/code> which can hold duplicate values and is ordered.<\/p>\n\n\n\n
>>> empty_set = set()\n>>> duplicate_list = [1,1,2]\n>>> set_with_values = {1,1,2}\n>>> print(duplicate_list)\n[1, 1, 2]\n>>> print(set_with_values)\n{1, 2}\n>>> set_with_values[0]\nTraceback (most recent call last):\n  File \"<stdin>\", line 1, in <module>\nTypeError: 'set' object does not support indexing<\/pre>\n\n\n\n
In the above example we see that duplicates cannot exist in a set<\/code> and that we also cannot access items by index in a set<\/code> . We use a set<\/code> when we specifically need the property of no duplicate values. We can compute all the normal set operations:<\/p>\n\n\n\n
>>> one_set = {1,3,4,5}\n>>> another_set = {4,5,8}\n>>> one_set | another_set # this is union\n{1, 3, 4, 5, 8}\n>>> one_set.union(another_set) # another way\n{1, 3, 4, 5, 8}\n>>> one_set & another_set # this is intersection {4, 5}\n>>> one_set.intersection(another_set) # another way {4, 5}\n>>> one_set - another_set # set difference\n{1, 3}\n>>> one_set.difference(another_set) # another way {1, 3}<\/pre>\n\n\n\n
A tuple<\/code> like a list<\/code> supports access by index and allows duplicate values. So it is very similar to a list. However, a list’s value can be changed (it is mutable<\/strong> ) while a tuple is immutable<\/strong>.<\/p>\n\n\n\n
>>> empty_tuple = ()\n>>> empty_tuple.append(1)\nTraceback (most recent call last):\n  File \"<stdin>\", line 1, in <module>\nAttributeError: 'tuple' object has no attribute 'append'\n>>> tuple_of_values = (1,2.3,\"informatics\")\n>>> list_of_values = [1,2.3,\"informatics\"]\n>>> print(tuple_of_values[1])\n2.3\n>>> list_of_values[-1] = \"i427\" # we can change items in a list \n>>> print(list_of_values)\n[1, 2.3, 'i427']\n>>> tuple_of_values[-1] = \"i427\"\nTraceback (most recent call last):\n  File \"<stdin>\", line 1, in <module>\nTypeError: 'tuple' object does not support item assignment<\/pre>\n\n\n\n
We can see that we can use [] <\/code>to access a value in a tuple by its index, but we cannot change that value. Likewise, we cannot add values to a tuple via append (or extend for that matter, even though it\u2019s not shown). If we need the property of immutability, then we use a tuple. In I427, we will see that rows from a database are returned to us as tuples. And then we can recognize the word tuple from I308 and connect the concept of collection of values to row in a table in a database.<\/p>\n\n\n\n
Traversing sets and tuples<\/h3>\n\n\n\n
Sets and tuples can be traversed exactly like lists. A dummy variable in a for loop can take on each value in the collection in turn. Tuples have an ordering, so a tuple would be reversed in order of index. Sets do not have an order, to the items in a set occur in no particular order. If we need to traverse items in a certain order, we would need to sort the set’s values, for instance.<\/p>\n\n\n\n
>>> tuple_of_days = (\"Friday\",\"Saturday\",\"Sunday\")\n>>> for day in tuple_of_days:\n...     print(day)\n...\nFriday\nSaturday\nSunday<\/pre>\n\n\n\n
Here is a similar example using a set.<\/p>\n\n\n\n
>>> set_of_days = {\"Friday\",\"Saturday\",\"Sunday\"}\n>>> for day in set_of_days:\n...     print(day)\n...\nSunday\nFriday\nSaturday<\/pre>\n\n\n\n
Once again, one thing worth noting is that the ordering of sets is not preserved. This is not a guarantee that the set class provides the programmer.<\/p>\n\n\n\n
Dictionaries<\/h2>\n\n\n\n
Dictionaries are a built-in data type in Python that provide some functionality of a hash table (the Java HashMap class) but are more properly called an associative array. One possible conceptual model of a dictionary is that it is a list that is indexed by an arbitrary key rather than an integer number from the sequence 0, 1, … length of data structure minus one. So the basic access pattern for a dictionary is dict object_name[key]<\/code> which looks just like accessing a list item. The indication that dict object is a dictionary and not a list comes when the object is instantiated; a list object is initialized with [ ]<\/code> while for a dictionary you use { }<\/code>, i.e. dict object_name = { }<\/code>. With a list, we insert items at a desired index. For instance append()<\/code> builds up a list item by item with the next index depending on the number of items in the list so far. Other methods we haven\u2019t covered let you insert items at other places in a list, but there is a performance penalty to do this. (Think back to linked lists in I210\/I211 where one must traverse a list to find a given position.) Dictionaries do not incur this performance penalty, so they provide fast lookup and insertion of arbitrary keys. So with a dictionary, we can add items in arbitrary order, and adding each item will be similarly efficient.<\/p>\n\n\n\n
>>> list_object = [1, 2, 3]\n>>> dict_object = {} # empty dictionary\n>>> dict_object[50] = 1 # assign the value 1 to the key 50\n>>> dict_object[0] = 2 # assign the value 2 to the key 0\n>>> dict_object[\"i427\"] = \"Search informatics\" # dictionaries keys don't have to be an integr >>> list_object[0]\n1\n>>> dict_object[0]\n2\n>>> list_object[1]\n2\n>>> dict_object[1]\nTraceback (most recent call last):\n  File \"<stdin>\", line 1, in <module>\nKeyError: 1\n>>> dict_object[50]\n1\n>>> dict_object[50] = 2 # overwrite the value for key 50<\/pre>\n\n\n\n
Note how easy adding an object to the dictionary is by simple assignment, and note the error message we receive if we try to access a non-existent key. If we evaluate dict object we see that it\u2019s presented as a comma-separated list of items with each item being of the form key:value. There is nothing special about the ordering of the key:value pairs, and there is no guarantee they will be presented to the programmer in any particular order. All the collection types we have seen (list, tuple, set, and dict) hold collections of items, but in all of them except dict an item is a single value. In a dict the item is a pairing of two things\u2014the key and the value.<\/p>\n\n\n\n
>>> dict_object\n{0: 2, \"i427\": \"Search informatics\", 50: 2}<\/pre>\n\n\n\n
At this point, it\u2019s important to remember that the \u201cindices\u201d (really the keys) don\u2019t have to be integers; they can be virtually any object (technically any object that\u2019s hashable, but that\u2019s beyond our scope here). So, as an example, let\u2019s use a dictionary to hold some data in a more typical situation. Let\u2019s say we have 5 character strings that act as primary keys for a table with 3 other columns of data \u2013 the first 2 of which are floating point while the last is a string, and also let\u2019s store the column names in our dictionary:<\/p>\n\n\n\n
>>> our_table = {}\n>>> our_table['column names'] = ['age','salary','name']\n>>> our_table['12345'] = [20.0,15000.,'Kimmer']\n>>> our_table['45678'] = []\n>>> our_table['45678'].append(25.5)\n>>> our_table['45678'].append(50000.)\n>>> our_table['45678'].append('Ramachandran')<\/pre>\n\n\n\n
This example shows that dictionaries can hold most anything. The values can be most any old object including lists. The second value is in fact begun as an empty list, and then we keep adding to it. Every time after the initial invocation of our table[\u201945678\u2019]<\/code> that initializes that value, invoking it returns out list to us, and with that list object, we\u2019re free to use any of the list methods we need including appending to add items to the value. Values are not immutable; they are mutable and can be changed.<\/p>\n\n\n\n
In the example above, the table of data was stored with one row per dictionary entry indexed by the key of the table. This is almost always a good way to store tabular data. The clunky part of the implementation above is that the column names are stored in their own dictionary entry, and column names aren\u2019t really an actual row in a table. They\u2019re metadata instead of data in the table. So let\u2019s fix that. If every row in the table has three values for each column, we could explicitly indicate that for each row.<\/p>\n\n\n\n
>>> our_new_table = {}\n>>> kimmer_row = {'age' : 20.0, 'salary' : 15000 ,'name' : 'Kimmer'}\n>>> dr_r_row = {'age' : 25.5, 'salary' : 50000 ,'name' : 'Ramachandran'}\n>>> our_new_table['12345'] = kimmer_row\n>>> our_new_table['45678'] = dr_r_row<\/pre>\n\n\n\n
So in this case we have made a dictionary of dictionarys instead of a dictionary of lists. Either is viable, but the advantage of the second approach is that we don\u2019t have to remember that the \u2019age\u2019 column is first, \u2019salary\u2019 second, etc. We can access columns by name which makes our programs considerably easier to write and to read. The readability of a program doesn\u2019t seem that important to many students at first, but once you realize large programs are written in pieces at many different times, the ability to come back to a partially completed program after a big gap of time and quickly get back up to speed with it is invaluable. So here\u2019s are good and not-so-good approacesh to accessing data in a table compared:<\/p>\n\n\n\n
>>> print(our_new_table['12345']['name'])\nKimmer\n>>> print(our_table['45678'][2]) # we \"just have to know\" that the name is stored third. It's less effective.<\/pre>\n\n\n\n
In either case, students need to come to terms with the idea of multiple indexing (the need for 2 or more [][] <\/code>when accessing items in the table). For tables, which are 2D data structures, you have to specify rows and columns. We store tables as collections of rows, so the row index is first, and then we access each row by column. So the indexing is [row][col]<\/code>, and we can read that from left to right. The first access is row and the second is col. In even more detail, when we access our new table by row via our new table[\u201912345\u2019] <\/code>we need to be thinking<\/p>\n\n\n\n
  • our_new_table <\/code>is a dict<\/code>, so we access it by key<\/li>
  • our_new_table[\u201912345\u2019] <\/code>returns the value associated with that key.<\/li>
  • The value is another dict<\/code> , so (our_new_table[\u201912345\u2019])<\/code> can be used in the same way a plain old dictionary object can<\/li>
  • (our_new_table[\u201912345\u2019])[\u2019name\u2019]<\/code> will then get the value associated with the \u2019name\u2019<\/code> key in our dictionary containing the row<\/li>
  • But the () <\/code>are unnecessary and detract from readability, so we should instead write it as our_new_table[\u201912345\u2019][\u2019name']<\/code> The same reasoning would apply to our table[\u201945678\u2019][2]<\/code> except (our table[\u201945678\u2019]<\/code>) is a list<\/code> and we must access it by index, so the next thing in []<\/code> must be a numerical list index.<\/li><\/ul>\n\n\n\n
    Traversing a dictionary<\/h3>\n\n\n\n
    If we use the same for-loop approach we\u2019ve used so far with lists, tuples, and sets, we\u2019ll end up with the keys of the dictionary stored in the dummy variable during each iteration. Note that the teams have changed for 2 out of the 3 players since this code was written. NBA churn, baby!<\/p>\n\n\n\n
    >>> dict_of_teams = { \"LeBron\" : \"Lakers\", \"KD\" : \"Warriors\", \"CP3\" : \"Rockets\" }\n>>> for key in dict_of_teams:\n...     print(key)\n...\nLeBron KD\nCP3<\/pre>\n\n\n\n
    Of course, if we have the keys, we can use them to access the values:<\/p>\n\n\n\n
    >>> for key in dict_of_teams:\n...     print(key)\n...     print(dict_of_teams[key])\n...\nCP3\nRockets\nLeBron\nCavs\nKD\nWarriors<\/pre>\n\n\n\n
    But such code is not very Pythonic. The root of the problem is that dictionaries are more complicated since they involve not one value per item in the collection but two\u2013the key and the value. The first example is good. If we only need the keys, that is how we traverse a dictionary. If we need the key and the value, then, Python provides an items()<\/code> function that returns both in dummy variables for every iteration.<\/p>\n\n\n\n
    >>> for key, value in dict_of_teams.items():\n...     print(key)\n...     print(value)\n...\nLeBron\nLakers\nKD\nWarriors\nCP3\nRockets<\/pre>\n\n\n\n
    Finally, it is worth mentioning that if we don\u2019t need the keys, we shouldn\u2019t bother to access them. Python also provides us with a values()<\/code> function in this case, but it\u2019s much less commonly-used than items()<\/code>. Incidentally, there\u2019s also a keys()<\/code> function that is equivalent to the first example. It\u2019s an antipattern<\/strong>, so it\u2019s not illustrated here.<\/p>\n\n\n\n
    >>> dict_of_teams = { \"LeBron\" : \"Lakers\", \"KD\" : \"Warriors\", \"CP3\" : \"Rockets\" }\n>>> for value in dict_of_teams.values():\n...     print(value)\n...\nLakers\nWarriors\nRockets<\/pre>\n\n\n\n
    To achieve this form of iteration, the .items()<\/code> member function of the dict class yields a 2-item tuple just like enumerate()<\/code> did with a list. On the other hand the member functions .keys()<\/code> and .values()<\/code> return either desired part of each dictionary entry. Referring to the first dictionary example, we can see that it is not necessary to invoke .keys()<\/code> :<\/p>\n\n\n\n
    >>> dict_of_teams = { \"LeBron\" : \"Lakers\", \"KD\" : \"Warriors\", \"CP3\" : \"Rockets\" }\n>>> for key in dict_of_teams:\n...     print(key)\n...\nLeBron\nKD\nCP3\n>>> for key in dict_of_teams.keys():\n...     print(key)\n...\nLeBron \nKD\nCP3<\/pre>\n","protected":false},"excerpt":{"rendered":"
    Python has a variety of built-in data structures that we will use in this course\u2014lists, tuples, sets, and dictionaries. All of these are similar in that they represent collections of …<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"featured_image_src":null,"featured_image_src_square":null,"_links":{"self":[{"href":"https:\/\/enter77.ius.edu\/cjkimmer\/wp-json\/wp\/v2\/pages\/524"}],"collection":[{"href":"https:\/\/enter77.ius.edu\/cjkimmer\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/enter77.ius.edu\/cjkimmer\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/enter77.ius.edu\/cjkimmer\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/enter77.ius.edu\/cjkimmer\/wp-json\/wp\/v2\/comments?post=524"}],"version-history":[{"count":12,"href":"https:\/\/enter77.ius.edu\/cjkimmer\/wp-json\/wp\/v2\/pages\/524\/revisions"}],"predecessor-version":[{"id":538,"href":"https:\/\/enter77.ius.edu\/cjkimmer\/wp-json\/wp\/v2\/pages\/524\/revisions\/538"}],"wp:attachment":[{"href":"https:\/\/enter77.ius.edu\/cjkimmer\/wp-json\/wp\/v2\/media?parent=524"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}