tutorial | Cody VanZandt

Better Coding Through Shakespeare

Sat, 15 Sep 2018 00:00:00 +0000

Shakespeare, as Ben Jonson reminds us, is “not of an age but for all time.” So, naturally, he has lots of good advice on computer programming.

This 7 part series draws on Shakespeare’s plays and Robert C. Martin’s Clean Code to illustrate some essential programming practices that rarely get covered in conventional “Learn to Code!” contexts. By the end of this post, you’ll have quality answers to some of programming’s toughest questions. What separates good variable names from bad ones? How big should functions be? When are comments necessary?

This blog post answers all of these questions (and more!) with level-headed pragmatism and a sizeable dollop of Shakespearean humor. So, sit back, relax, and enjoy…

Better Coding Through Shakespeare

Act I: Naming Things. Or, Romeo Is a Dreadful Programmer

In Romeo and Juliet, Romeo famoulsy opines…

What’s in a name? That which we call a rose

By any other name would smell as sweet.[1]

Now that might be a fine philosophy for Montagues and Capulets, but it’s a dreadful way to approach programming.

Names matter.

Well-named variables, functions, and classes can significantly improve the readability and maintainability of a program.

For example, check out this function:

def get_result(l):
result = list()
for i, j in l:
if j > 40:
result.append(i)
return result

To quote Robert Martin,

The problem isn’t the simplicity of the code but the implicity of the code. [2] You might know what this code does, but you have no idea why it’s doing it. What’s the significance of the number 40? What’s i? What’s j? What are the inputs? The outputs?

You just don’t know.

But with a little thoughtful renaming, everything becomes clear:

OVERTIME_CUTOFF = 40
def get_employees_with_overtime(employees_and_hours):
employees_with_overtime = list()
for employee_name, hours_worked in employees_and_hours:
if hours_worked > OVERTIME_CUTOFF:
employees_with_overtime.append(employee_name)
return employees_with_overtime

Ah, that’s better! Now you know exactly what this function and all of its variables are for.

See Names as Opportunities to Add Value

This is dreadful:

s = 50000

This is better:

salary = 50000

But this is best:

base_salary= 50000

i,j,k (and Friends) Are Only For Indices

Let’s say you want to iterate through a list of office locations for a Texas-based company:

office_locations = ["Houston", "Austin", "Dallas"]

This is awful. i adds no value here; it can only confuse you.

for i in office_locations:
pass

This is vaguely acceptable, but only if you know that (1) enumerate returns indices and (2) i is a conventional name for indices

for i, location in enumerate(office_locations):
pass

These two options are likely the best. These variable names help communicate the intention of the code (to iterate through locations) and even make the semantics of “enumerate” clearer.

for location in office_locations:
pass
for loc_index, location in enumerate(office_locations):
pass

Don’t Add Types If They Don’t Add Information

Appending type information to variables is usually a poor idea. Variable names can lie – there’s nothing to stop a variable named this_is_totally_a_list from being (or becoming!) something other than a list. So, don’t add typing information through variable names. Instead, use your language’s type system (or type hint system!) – it’s there for a reason.

These variables, for example, are probably poorly named:

employee_name_string = "Robin"
employee_name_list = ["Robin", "Sam", "Alex"]

Make Sure You Can Pronounce It

If you need a variable for a company street address, don’t do this:

cmpy_strt_adr = "100 Foo Street"

No one wants to stumble through pronouncing “cumpy stert adder”. Be kind to yourself and your coworkers – create names that aren’t torturous to pronounce.

Variables and Classes are Nouns. Functions and Methods are Verbs.

Classes model things; methods model actions. Nouns represent things; verbs represent actions. There’s a natural division here, and you should respect it. Use nouns to name your classes; use verbs to name your methods.

Don’t do this:

class GetEmployeeInfo(object):
def years_of_service_calculator(self):
pass

Instead, do this:

class Employee(object): # GOOD!
def get_years_of_service(self): # GOOD!
pass

Pick Your Conventions and Stick with Them

If you use snake_case for methods, use snake_case. If you prefer camelCase, use camelCase.

If you like using the “get” prefix, that’s great – don’t stray into “fetch” or “acquire” unless you have an very convincing use-case.

Inconsistent conventions magnify the cognitive burden of reading code; consistent conventions ease that burden. This might seem trivial, but I assure you it is not. Programming is hard enough without cognitive self-sabotage.

So, definitely don’t do this:

class Employee(object):
def getName(self):
pass
def fetch_title(self):
pass
def Return-Salary(self):
pass
def aCqQiReMaNaGeR(self):
pass

Act II: Functions. Or, Hermia Would Make a Good Function

In A Midsummer Night’s Dream, Helena says of Hermia…

And though she be but little, she is fierce.[3]

Just the qualities we want in a function: little and fierce.

Programs made up of small, powerful functions are easier to understand, easier to test, and easier to maintain.

Check out this big ol' function. No need to get too involved with it. It is, after all, fake. But glance through it and get the gist.

def promote_employee(employee, new_title):
# give the employee his new title, adding 'Senior' or 'Junior' if necessary
if employee.years_of_service > 10:
fully_udated_title = "Senior " + new_title
elif employee.years_of_service < 2:
fully_udated_title = "Junior " + new_title
employee.title = fully_udated_title
# compute the employee's new salary
new_salary_bracket_min, new_salary_bracket_max = human_resources.get_salary_bracket(new_title)
if employee.salary <= new_salary_bracket_min:
new_salary = new_salary_bracket_min
elif employee.salary >= new_salary_bracket_max:
new_salary = new_salary_bracket_max
else:
new_salary = ( new_salary_bracket_min + new_salary_bracket_max ) / 2.0
employee.salary = new_salary
# compute the employee's new 401k matching
if human_resources.is_role_401k_eligible(new_title):
new_max_401k_matching = human_resources.get_max_401k_matching(new_title)
min_401k_matching = human_resources.get_company_minimum_401k_matching()
if employee.matching_401k == 0:
new_matching_401k = min_401k_matching
else:
new_matching_401k = min(new_max_401k_matching, employee.matching401k + 0.01)
else:
new_matching401k = 0
employee.matching_401k = new_matching_401k
# compute the new stock compensation
stock_level_1, stock_level_2, stock_level_3 = human_resources.get_stock_levels(new_title)
if employee.tenure > 20:
new_stock_total = stock_level_3
elif employee.tenure > 10:
new_stock_total = stock_level_2
else:
new_stock_total = stock_level_1
employee.stock = new_stock_level
return employee

That’s one long function. It isn’t terribly easy to read, the logic of each individual part obfuscates the function’s larger goal, and there are control-flow statements everywhere. The comments help a little, but they shouldn’t be necessary.

In short, this function isn’t short, and it isn’t clear.

Compare it with these beautiful little functions:

def promote_employee(employee, new_title):
update_title(employee, new_title)
update_salary(employee, new_title)
update_401k_matching(employee, new_title)
update_stock(employee, new_title)
return employee
def update_title(employee, new_title):
if employee.years_of_service > 10:
new_title = "Senior " + new_title
elif employee.years_of_service < 2:
new_title = "Junior " + new_title
employee.title = new_title
# imagine that the rest of the new "update" functions are similarly defined below

Now that is readable!

It’s immediately clear, even at a glance, what is involved in promoting an employee. The details of updating title, salary, 401k, and stock no longer obfuscate the larger purpose of promote_employee. If anyone cares about the finer details of, say, title updating, they can go look in update_title.

There are three tricks here: (1) functions should be small, (2) do only one thing, and (3) operate at a single level of abstraction. The next sections cover each trick in detail.

Function Trick #1: Functions Should Be Small

To quote Robert C. Martin on it,

The first rule of functions is that they should be small. The second rule of functions is that they should be smaller than that. [5]

Very clever. But how small is “smaller than that?”

Honestly, there isn’t a universally-accepted measure of “small.” Here are some statistics from a Ruby codebase written by Martin Fowler. Fowler is something of a thought-leader in software engineering theory, so one could do worse than emulating him.

59% of his functions are less than three lines
75% of his functins are less than five lines
93% of his functions are less than ten lines [6]

Now those are some small functions! Granted, Fowler is both (1) particularly devoted to the idea of small functions and (2) writing in Ruby, a particularly terse language. Your definition of small need not be the same as Fowler’s; you should treat these numbers as shaggy heuristics rather than a statements of Truth.

As a general rule, if your functions start to exceed 10 lines, you should take a step back and ask yourself: does this function really need to be this big? The answer might very well be “Yes!”, but it’s probably “No…”

Luckily, if you follow trick #2, your functions will naturally remain small.

Function Trick #2: Functions Should Do One Thing

To quote Robert C. Martin again,

Functions should do one thing. They should do it well. They should do it only. [7]

This, of course, begs the question: how does one know if a function is doing only one thing?

This is relatively straightforward: try to extract out another meaningful function – if you’re successful, then your function was doing more than one thing. To illustrate this, take another look at promote_employee:

def promote_employee(employee, new_title):
update_title(employee, new_title)
update_salary(employee, new_title)
update_401k_matching(employee, new_title)
update_stock(employee, new_title)
return employeee

Keeping in mind the idea that functions should do only one thing, is there another function that could be extracted from promote_employee?

There’s an argument to be made that update_salary, update_401k_matching, and update_stock might be extracted into a new function: update_compensation. This new function does one thing – it updates an employees compensation.

Take a look:

def promote_employee(employee, new_title):
update_title(employee, new_title)
updated_compensation(employee, new_title)
return employee
def update_compensation(employee, new_title):
update_salary(employee, new_title)
update_401k_matching(employee, new_title)
update_stock(employee, new_title)

Now those are some good looking functions! Try as you might, there just doesn’t seem to be any way of extracting additional meaningful functions from either promote_employee or update_compensation. These functions have been divided up into their smallest reasonable logical components, and as a result, they are small, readable, easy to test, and easy to maintain.

Function Trick #3: Functions Should Operate at a Single Level of Abstraction

This final function trick is, unsurprisingly, a little more abstract. What exactly is a “level of abstraction”? And why should functions only operate at one of them?

Put simply, a level of abstraction is a level of detail. High levels of abstraction are light on details; low levels of abstraction of heavy on details.

Consider update_compensation:

def update_compensation(employee, new_title):
update_salary(employee, new_title)
update_401k_matching(employee, new_title)
update_stock(employee, new_title)

This function operates at a relatively high level of abstraction – it’s concerned with what updating compensation entails, not with the details of how compensation updates are carried out. If, for example, you decided to include the specific details of update_stock in update_compensation, you would end up with a rather muddled looking update_compensation function:

def update_compensation(employee, new_title):
update_salary(employee, new_title)
update_401k_matching(employee, new_title)
stock_level_1, stock_level_2, stock_level_3 = human_resources.get_stock_levels(new_title)
if employee.tenure > 20:
new_stock_total = stock_level_3
elif employee.tenure > 10:
new_stock_total = stock_level_2
else:
new_stock_total = stock_level_1
employee.stock = new_stock_level

update_compensation is now operating at two levels of abstraction, two levels of detail. It contains high-abstraction, low-detail salary and 401k updates. But it also contains a very low-abstraction, high-detail stock update. This update just doesn’t fit. It makes update_compensation harder to read, harder to describe, and harder to understand in the context of the employee-promoting process.

So after you code up a function, ask yourself if it’s operating at a single level of abstaction. If it isn’t, you’ve likely discovered an excellent opportunity to make your code easier to read, explain, and maintain.

Act III: “Come hither, come hither. How did this argument begin?” Or, Arguments Confuse Don Adriano (And Everyone Else)

Arguments complicate your code. There’s no way around it.

Arguments make testing, understanding, and debugging code more difficult, and that difficulty grows exponentially in the number of arguments.

Naturally, our friend Robert C. Martin agrees:

The ideal number of arguments for a function is zero (niladic). Next comes one (monadic), followed closely by two (dyadic). Three arguments (triadic) should be avoided where possible. More than three (polyadic) requires very special justification — and then shouldn’t be used anyway. [9]

Martin takes a pretty hard line against arguments. There are, in fact, some excellent reasons to write functions with more than three functions. Heavily-used public-facing objects (like Python’s pandas dataframe) intentionally heap loads of functionality into a fantastically convenient interface that necessarily requires a larger number of arguments.

But! Martin is right in the general case. Testability, readability, and maintainability do tend to decrease as the number of arguments increases. Accordingly, you should do your best to limit the number of arguments in your functions and classes.

Here are some tricks for tamping down your arguments!

Argument Trick #1: Make Your Arguments Classy

You might be writing a function like this…

def manipulate_3D_point(x, y, z):
# Do something nifty with a point sitting in three dimensions.
pass

Any function that needs to know about a point sitting in 3D space will need at least three arguments: x, y, and z. But that’s not a stellar start to writing functions with only a few arguments.

What’s a programmer to do?

Well, since x, y, and z are all necessary to make a 3D point, they can (and should!) be bound up in a 3D point class!

class Point_3D(object):
def __init__(self, x, y, z):
self.x = x
self.y = y
self.z = z
def manipulate_3D_point( a_3D_point ):
# Do something nifty with something a point sitting in three dimensions.
pass

Much better! You’re function has all the information it needs, you’ve minimized your argument list, and I’d even argue that you added significant clarity to your code by adding a simple little class. (Note: for Pythonic fun with classes that simply bind data together, check out Python’s namedtuple.

Granted, this example is pretty obvious. But the point remains (pun intended): when your argument list starts to grow, some of your arguments are likely related enough to make a coherent class.

Argument Trick #2: Make Sure Arguments Don’t Come OUT of Functions

Arguments are confusing enough. But arguments that come out of functions are even worse.

At this point you should be a little confused. Don’t arguments go into functions by definition? Why yes. They do. But if you’re not careful, they can surprise you (and everyone else) by coming back out of your functions. Let me illustrate.

Imagine that you’re reading some (icky) code, and you stumble across this little gem:

text = extend_text( text, extend_value )

This line of code looks like it probably…

Takes a text and another value as arguments
Extends the text using extend_value
…and then returns text again.

So text goes in, and then comes back out. Yuck.

This works significantly better as

text.extend( value )

Ah, much better! value goes in, text comes out, and no one is surprised.

Argument Trick #3: Flags are for Countries, Not for Programmers

Programmers love passing flags to functions.

Like so…

class SomeCrazyData(object):
def output_data(self, as_CSV):
if as_CSV:
# Output the data as CSV
return some_CSV
else:
# Output the data as JSON
return some_JSON

Look how simple! Now you can get output_data to return the data in CSV or JSON format. All you have to do is pass a boolean flag and WHAM! You’re done!

But this function makes me feel really uneasy.

as_CSV is a boolean. If its value is True, then it returns a CSV. If its value is the opposite of True, then it returns… Uh… Well I guess it returns the opposite of a CSV. Whatever that is. I haven’t the foggiest idea.
Functions do one thing! But… This function does two things. One for the True case and another for the False case. That’s unfortunate.
The function’s name and arguments don’t tell me what I can use the function for! To figure that out, I have to rely on good documentation (Ahahahahaha! Yeah right…) or look around inside the function until I figure it out.

To recap, we have a function with an ambiguous argument that does two things without making it clear from its definition what those two things are.

Yuck.

It’s a much better idea to do something like this:

class SomeCrazyData(object):
def output_data_as_CSV(self):
# Output the data as CSV
return some_CSV
def output_data_as_JSON(self):
# Output the data as JSON
return some_JSON

Do yourself a favor: don’t use flags unless you absolutely have to.

Argument Trick #4: Leave “None” Alone!

None is so tempting. But it’s usually a sub-optimal idea.

It is a value that represents the absence of any value. And that alone should make you feel weird.

You cannot iterate through None. You cannot cannot use it as a function. It has no attributes, and it has no methods. All None will ever do for you is raise the exception “AttributeError: ‘NoneType’ object has no attribute…”

If you return it from a function, it will force every caller of your function to perform None-checks. If they forget a check, they risk an AttributeError.

If you pass it into a function, (especially as a default argument), you’ll obfuscate your logic with None-checks and doom your helper functions to the same fate.

And yet, None still gets used because it is convenient. It has no features: all you can do with it is ask “Are you None?”

Given its simple featurelessness, folks use None to store all kinds of random information: Did I ever set this variable? Did I call this function? Did I succeed in some task? All sorts of things.

They’re all a mistake. There is always a better way to signal your intention than with a None.

You could…

Raise an exception (potenially in a try-except block) to signal that something invalid happened
Ditch the default argument. If an argument doesn’t have a sensible default, then why are you trying to force one on it?
Use a sensible, non-mutable default: a custom “empty” class or an enum.
Re-architect your functions and logic so that they don’t need None. Your code will almost certainly be more robust as a result.

Just try not to use None.

If you’re still not convinced, consider for a moment that Turing award-winning computer scientist Tony Hoare believes that None (aka: the null refernece) is a mistake. And he invented it.

Act IV: “Forgive the comment that my passion made / Upon thy feature” [11] Or, King John Regrets Commenting and You Should Too

Comments are weird.

Countless programmers extol the virtues “well-commented” code, but no two people ever seem to have the same idea of what “well-commented” means. Now why is that? As you might have come to expect, we’re going to turn to Robert C. Martin for the answer:

The proper use of comments is to compensate for our failure to express ourself in code. Note that I used the word failure. I meant it. Comments are always failures. [12]

It is worth repeating: comments are always failures. We fail to express ourselves clearly in code, and so we leave a comment behind to mark that failure. Over time, that comment slowly forgets its purpose as the surrounding system evolves. And then one day, the comment looks up and realizes that it no longer means anything to anyone. It has no purpose. All it can do is wait to be deleted or replaced.

Almost brings a tear to your eye, doesn’t it?

If only our programming languages were more expressive – if only we were more expressive! But they aren’t, and we’re not. So there are still a few kinds of neccesary (but necessarily evil!) comments.

Dire Warnings

# DONT CHANGE THIS NUMBER. If it isn't EXACTLY equal to 4, then - believe it or not - the San Andreas Fault will violently hemorrhage
# and cause San Francisco to topple helplessly into the Pacific.
b = 4

Occasionally, things like this happen. Systems aren’t always as robust as we’d like them, and programmers who’ve erred leave warnings behind. They’re often valuable, and not to be dismissed without investigation.

Explaining Interfaces You Don’t Own

# This only works if you pass in the username in Pig Latin...
fetch_BookFace_messages(username="odycay anzandtvay")

We hope that all public APIs are well-documented and entirely unastonishing. But that’s just not the case. Ocassioanlly, Alice-in-Wonderland-esque functions climb out of the rabbit hole and make their way into our systems. We can sputter and swear and jump up and down all we’d like, but there’s often very little we can do to change wonky code that we don’t own. All we can do is leave a comment and hope for the best.

Admittedly, the wonky code might one day change and our comment will become unnecessary. But ‘til then, a few words might save our colleagues loads of time and energy.

Explaining the Truly Esoteric

Occasionally, you’ll find yourself coding in the footsteps of someone with a skill set very different from your own. That previous coder might be, for example, a math PhD. And although she has long left the firm, she did provide a nice equation to help you understand her code:

\begin{align*} &a=12+7 \int_0^2 \left( -\frac{1}{4}\left(e^{-4t_1}+e^{4t_1-8}\right) \right)
\end{align*}

Alas, your degree isn’t in math, so you don’t haveno hope of making sense of this.

However!

Your erstwhile colleague also provided a nice, English description of this equation’s purpose and implementation in a comment:

# Here's exactly what the equation does and why...
# <insert clear explanation here>

Now you can impress your fellow developers at the next meeting with your lucid descriptions of complex computations.

Hurray!

Licenses, Test Files, Permissions, Primary Maintainers, and Other Necessities

"""
Primary: Cody VanZandt
Permission Group: Blah Blah and Foo
Tests: /tests/ThingDoer.py
License: Creative Commons 4.0 Attribute
"""
def a_very_important_function():
# do some very important computational business

This stuff is pretty important in systems with lots of developers. Some folks might even scrape and mine a large collection of it for its secrets.

Not all such file statistics are useful (things like version number are generally best left to version control software), but a lot of them are, so they’re ofen worth including.

Act V: “You are too blunt. Go to it orderly.” Or, Petruchio Needs Better Formatting

Poor formatting can hamstring your readers just as easily as poor design.

If no one knows where to look for anything, the task of understanding what’s actually happening grows more difficult.

You (and everyone who reads your code) will be grateful if you spend a litte time formatting your code to be readable and unsurprising.

How Long Should a File Be?

The short answer? I don’t know.

Just long enough and not any longer, I think.

1000 lines is usually too many, and a single line is usually too few. In truth, file length isn’t the sort of thing you should optimize. You should instead ask yourself “In one sentence, what is this file doing?” If you can’t answer that question without a slough of commas and conjuctions, then you’re file is too long. See, files - just like functions - should only do one thing.

Where Should I Put Things?

Ah, now this question is much easier!

I am, unsurprisingly, a huge fan of Robert C. Martin’s answer…

We would like a source file to be like a newspaper article. The name should be simple but explanatory. The name, by itself, should be sufficient to tell us whether we are in the right module or not. The topmost parts of the source file should provide the high-level concepts and algorithms. Detail should increase as we move downward, until at the end we find the lowest level functions and details in the source file. [13]

That is a fantastic simile: a source file is a like a newspaper article.

Let’s look an example of this done really wrong:

def add_report_content(empty_report):
content = get_report_content()
full_report = empty_report + content
return full_report
def make_unformatted_report():
return "An unformatted report"
def format_report(unformatted_report):
return unformatted_report.format()
def make_report_base():
unformatted_report = make_unformatted_report()
return format_report(unformatted_report)
def get_report_content():
return "Some content."
def make_report():
report_base = make_report_base()
finished_report = add_report_content(report_base)
return finished_report

How obvious is this program’s flow? Can you glance through it and tell me how these functions work together? Which functions depend on which other functions?

It’s not obvious.

Sure, you can probably look around and figure out that everything starts with make_report, but the file leads you on a merry chase - up and down, and up and down, and… Eh. It’s a mess.

Compare with this…

def make_report():
report_base = make_report_base()
finished_report = add_report_content(report_base)
return finished_report
def make_report_base():
unformatted_report = make_unformatted_report()
report_base = format_report(unformatted_report)
return report_base
def make_unformatted_report():
return "An unformatted report"
def format_report(unformatted_report):
return unformatted_report.format()
def add_report_content(empty_report):
content = get_report_content()
full_report = empty_report + content
return full_report
def get_report_content():
return "Some content."

Now that’s much better!

What does this file do? It’s right at the top: it makes a report with make_report.

You can follow the logic of make_report first through make_report_base and then through add_report_content. Each of which is located (along with its dependent functions) in order right below make_report.

It’s just like a well structured essay:

Thesis (make_report)
- Topic 1 (make_report_base)
  - Support for Topic 1 (make_unformatted_report)
  - Support for Topic 1 (format_report)
- Topic 2 (add_report_content)
  - Support for Topic 2 (`get_report_content)

Do Not Line Up Your Assignments

Lastly, a minor and perhaps controversial suggestion.

Let’s say you have a big ol’ load of assignment statements…

def some_function():
a_second_long_variable = 5
short_var = 1
an_extremely_long_variable_name = 6
longer_variable = 3
also_short = 2
an_even_longer_variable = 4
return "Boo! Terrible!"

So you think to yourself,

“Gee, those staggered assignment statements sure look ugly. Let me fix that up…”

def some_function():
a_second_long_variable = 5
short_var = 1
an_extremely_long_variable_name = 6
longer_variable = 3
also_short = 2
an_even_longer_variable = 4
return "Boo! Terrible!"

Ah! That’s much better!

Except… it isn’t better at all. It’s worse.

Firstly, you missed the point. The problem isn’t that loads of staggered assignments right in a row look ugly. The problem is that you have loads of staggered assignments. Don’t try to hide that issue under a fresh coat of paint; fix it.

Secondly, you forgot what assignment statements are really about. You’ve drawn the eye away from the assignments and towards the giant list of values. But assignment is the important thing here, not the giant list of values.

And finally, this non-traditional assignment plays tricks on the mind. If I didn’t know better, I might think for a crazy second that short_var is equal to a large amount of whitespace followed by a 1.

Weird.

Act VI: “Doth not the object cheer your heart, my lord?” Or, Queen Margaret Loves Object-Oriented Programming

Objects are great; object-oriented programming is fantastic.

And while I don’t have the space in this series (let alone this section) to read you the riot act on object-oriented programming, I do have enough space to answer two of the most important questions on classes.

First up…

When Should You Use a Class?

There are many reasons to use classes!

Here’s a useful (though wildly incomplete) list:

To protect your data from accidental or incorrect manipulation
To keep data items close to the functions that act on them
To reduce code reuse
To model life in code with fidelity
To provide useful abstractions that make complex systems understandable
And more!

But at the end of the day, all of these reasons reduce down to one: you have a rag-tag set of variables (and optionally: functions) that would probably work better if you imposed a little structure.

So something like this…

def do_something(a, b, c):
pass
def do_something_else(a, b, c):
pass
def do_yet_another_thing(a, b, c):
pass

Usually works better like this…

class ABC_Thing(object):
def __init__(self, a, b, c):
self.a = a
self.b = b
self.c = c
def do_something(self):
pass
def do_something_else(self):
pass
def do_yet_another_thing(self):
pass

How Big Should Classes Be?

To quote Robert C. Martin one last time,

The first rule of classes is that they should be small. The second rule of classes is that they should be smaller than that.[15]

But, we will ask for one last time in this series: just exactly how small is small?

Unlike functions, it’s not productive to measure size in lines. A class might work with 20 (hopefully small) methods, or with two.

The only reliable way to corral the size of your classes is to look at how many responsibilities that class performs.

Because like functions, a class should only do one thing.

For example…

class Lamp(object):
def turn_on(self):
pass
def turn_off(self):
pass
def convert_to_AC(self, direct_current):
pass
def convert_to_DC(self, alternating_current):
pass

Now I’m no electrician, but I’d say that a lamp probably shouldn’t know anything about converting different currents. That should probably be in a class called… uh… whatever it is you call the thing that converts currents.

Whatever that thing is called, the point remains the same: you shoulnd’t be afraid to split up an overburdened class into a handful of smaller, cleaner classes.

Act VII: Final Words, Or What You Will

If you’ve made it all the way through Better Coding Through Shakespeare, I congratulate you. This series weighs in at a few thousand words and includes a fair bit of code. It’s not exactly light reading.

I hope that you’ve found this series instructive, but I’ll also accept that you believe I am…

“A most notable coward, an infinite and endless liar, an hourly promise breaker, the owner of no one good quality.” [16]

And you know, that’s fine by me.

I didn’t write this series to proselytize for my particular coding religion. I wrote this because these questions of functions, arguments, comments, formatting, and objects are all worth asking. It doesn’t matter that you’ve accepted my answers, only that you’ve considered the questions.

Thanks for reading, and good luck out there.

[ Exit, pursued by a bear. ] [17]

Works Cited

[1] Shakespeare, William. Romeo and Juliet. II.ii

[2] Martin, Robert C. Clean Code. 18.

[3] Shakespeare, William. A Midsummer Night’s Dream. III.ii

[4] Thanks, Mom!

[5] Martin, Robert C. Clean Code. 34

[6] Fowler, Martin. https://martinfowler.com/bliki/FunctionLength.html

[7] Martin, Robert C. Clean Code. 35

[8] Shakespeare, William. Love’s Labor’s Lost. III.i

[9] Martin, Robert C. Clean Code. 40

[10] Shakespeare, William. King John IV.iii

[11]Martin, Robert C. Clean Code. 54

[12] Shakespeare, William. The Taming of the Shrew II.i

[13] Martin, Robert C. Clean Code. 77

[14] Shakespeare, William. Henry VI II.

[15] Martin, Robert C. Clean Code. 136

[16] Shakespeare, William. All’s Well That Ends Well III.VI

[17] Shakespeare, William. Winter’s Tale III.iii

The Sorcerer's Code

Fri, 31 Aug 2018 00:00:00 +0000

We’re Going to Need a Little Magic

So there you are - mind racing, fingers blurring, the most beautiful Python in all the world coming to life on your screen. You’re in the zone!

Tasks fall to your indomitable will one right after another. Boom! Boom!

Suddenly, a new task appears on the horizon: you need a class to represent a big ol' hunk of words!

“Ha,” you scoff. “Easy!”

It takes mere seconds for you to lay out the core of the class.

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()

You give the class a try…

hunk = WordHunk("These are some words")
assert hunk.words == ["These", "are", "some", "words"]

…and it’s flawless, of course. You’re an absolute wizard on the keyboard today!

Confidently, you print out your new class:

hunk = WordHunk("These are some words")
print(hunk)

<__main__.WordHunk object at 0x7fb5684d5ef0>

Oh. Oh no.

That’s so… ugly!

You desperately wrack your brain for a solution, but nothing comes to mind. Could you maybe change the print function? Could you do something weird with stdout?

Ugh - those are terrible ideas.

You sag in your chair, and your flow state comes to an abrupt end.

Magician though you may have been, you’re going to need some real magic to get out of this mess.

Luckily, Python has you covered.

Allow me to introduce you to…

Magic Methods!

The `init` Method

You’re probably pretty familiar with this little number already.

By defining __init__, we can control how class instances are initialized.

We used this method just above when we dictated that WordHunk initializes instances by:

Converting a user-provided string into a list of individual words
And assigning that new list of words to the instance variable words

The `str` Method

__str__ allows us to define a “human-readable” representation of our class. Whatever __str__ returns is what print will return.

Armed with this new knowledge, let’s fix up WordHunk:

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
def __str__(self):
return str(self.words)

hunk = WordHunk("These are some words")
print(hunk)

['These', 'are', 'some', 'words']

Now that is magical!

Let’s just make sure that WordHunks are always represented in such a pretty way…

double_hunk = [hunk, hunk]
print(double_hunk)

[<__main__.WordHunk object at 0x7fb5684e30b8>, <__main__.WordHunk object at 0x7fb5684e30b8>]

Oh no. Not again!

The `repr` Method

Where __str__ fails, __repr__ excels.

__repr__ allows us to define a “machine-readable” representation of our class. Whatever __repr__ returns is what will be rendered when we use our class in a data structure or on the command line.

By convention, the return value of __repr__ should be valid Python code. Ideally, we should be able to recreate an instance of a class from that instance’s __repr__.

Let’s get to it!

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
def __str__(self):
return str(self.words)
def __repr__(self):
return "WordHunk('{}')".format(" ".join(self.words))

hunk = WordHunk("These are some words")
double_hunk = [hunk, hunk]
print(double_hunk)

[WordHunk('These are some words'), WordHunk('These are some words')]

Perfect! And as a quick double-check, we can get the __repr__ of our objects using Python’s repr function.

repr(hunk)

"WordHunk('These are some words')"

And if we evaluate our __repr__ as if it were valid Python code (because it is), then we should get back a real Python object:

eval(repr(hunk))

WordHunk('These are some words')

Ta da! A real WordHunk!

The `len` Method

Wouldn’t it be cool if we could take the length of a WordHunk using Python’s built-in len function?

len(hunk)

---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-11-d796a42c142a> in <module>()
----> 1 len(hunk)
TypeError: object of type 'WordHunk' has no len()

Well I sure think it’d be cool!

All we have to do is define the __len__ method. Whatever __len__ returns is how long our WordHunk will be.

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
def __str__(self):
return str(self.words)
def __repr__(self):
return "WordHunk('{}')".format(" ".join(self.words))
def __len__(self):
return len(self.words)

hunk = WordHunk("These are some words")
len(hunk)

That was pretty easy. It makes sense that the length of a WordHunkis equal to how many words it contains.

The `contains` Method

I think it would also be pretty nifty if we could check if a word is in a WordHunk.

Something like this:

hunk = WordHunk("These are some words")
"some" in hunk

---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-14-954425367c9d> in <module>()
1 hunk = WordHunk("These are some words")
----> 2 "some" in hunk
TypeError: argument of type 'WordHunk' is not iterable

To get this sort of behavior from our WordHunk, the simplest thing to do is define the __contains__ method.

__contains__ should take an additional argument (the item in question) and then return a boolean depending on whether or not that item is in our WordHunk.

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
def __str__(self):
return str(self.words)
def __repr__(self):
return "WordHunk('{}')".format(" ".join(self.words))
def __len__(self):
return len(self.words)
def __contains__(self, item):
return item in self.words

hunk = WordHunk("These are some words")
"some" in hunk

True

The `getitem` Method

Here’s the next awesome feature I think we should add to WordHunk:

hunk = WordHunk("These are some words")
hunk[0] # Should return: "These"

---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-17-2a1dee428c19> in <module>()
1 hunk = WordHunk("These are some words")
----> 2 hunk[0] # Should return: "These"
TypeError: 'WordHunk' object does not support indexing

To get WordHunk to support indexing, we need to define __getitem__.

__getitem__ should take an additional argument (a index of some sort) and returns the word located at that index.

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
def __str__(self):
return str(self.words)
def __repr__(self):
return "WordHunk('{}')".format(" ".join(self.words))
def __len__(self):
return len(self.words)
def __contains__(self, item):
return item in self.words
def __getitem__(self, key):
return self.words[key]

hunk = WordHunk("These are some words")
hunk[0]

'These'

The `setitem` Method

Now that we can get an item using bracket-notation, it would be nice if we could set an item using bracket notation.

Something like this:

hunk = WordHunk("These are some words")
hunk[2] = "different"
hunk

---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-20-0fdc933bef70> in <module>()
1 hunk = WordHunk("These are some words")
----> 2 hunk[2] = "different"
3 hunk
TypeError: 'WordHunk' object does not support item assignment

No surprises here; we’re going to use __setitem__ to implement this feature.

__setitem__ takes two arguments:

a key to use as an index
and a new value to write at that index

Note: there’s really no need for __setitem__ to return anything. I can’t think of anything reasonable for it to return.

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
def __str__(self):
return str(self.words)
def __repr__(self):
return "WordHunk('{}')".format(" ".join(self.words))
def __len__(self):
return len(self.words)
def __contains__(self, item):
return item in self.words
def __getitem__(self, key):
return self.words[key]
def __setitem__(self, key, new_value):
self.words[key] = new_value

hunk = WordHunk("These are some words")
hunk[2] = "different"
hunk

WordHunk('These are different words')

The `iter` Method

It’d be super useful if we could also iterate through our WordHunk one word at a time.

Something like this:

hunk = WordHunk("These are some words")
for word in hunk:
print(word)

These
are
some
words

Given how this guide has gone so far, you wouldn’t have expected that to work. Turns out, if you define __getitem__, Python will give you iteration for free. Tricky, tricky!

But what if you don’t want to define __getitem__, but you still want iteration?

Have no fear! Here are the nuts and bolts of how Python implements iteration.

We need to define two methods on WordHunk: __iter__ and __next__.

Here are the rules for __iter__ and __next__.

__iter__ needs to return an object that has a __next__ method.
__next__ is a method that returns the next item until there are no more items left. At that point it throws a special exception: the StopIteration exception.

Let’s start simple by defining __next__ on our WordHunk.

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
self.current_index = 0
def __str__(self):
return str(self.words)
def __repr__(self):
return "WordHunk('{}')".format(" ".join(self.words))
def __len__(self):
return len(self.words)
def __contains__(self, item):
return item in self.words
def __getitem__(self, key):
return self.words[key]
def __setitem__(self, key, new_value):
self.words[key] = new_value
def __next__(self):
if self.current_index < len(self.words):
next_word = self.words[self.current_index]
self.current_index += 1
return next_word
else:
raise StopIteration

There isn’t anything tricky going on here.

We now have a current_index variable to keep track of where we are in our word list.

When we call __next__, we check if current_index is a valid index in our word list.

If current_index is valid, we return the word at current_index in our list and increment current_index by 1.
If current_index is invalid, we raise a special exception, StopIteration.

SoWordHunk has a next method. …What should we do now?

Let’s think back to the requirements for making an object iterable.

The object must have an __iter__ method that returns something with a __next__ method.
The object’s __next__ method must return the next item until it runs out of items, then it throws.

Hang on… If WordHunk’s __iter__ method needs to return an object with a __next__ method, and WordHunk is an object with a __next__ method… then that means WordHunk’s __iter__ can just return… WordHunk.

Whoa. That’s trippy. But there it is!

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
self.current_index = 0
def __str__(self):
return str(self.words)
def __repr__(self):
return "WordHunk('{}')".format(" ".join(self.words))
def __len__(self):
return len(self.words)
def __contains__(self, item):
return item in self.words
def __getitem__(self, key):
return self.words[key]
def __setitem__(self, key, new_value):
self.words[key] = new_value
def __next__(self):
if self.current_index < len(self.words):
next_word = self.words[self.current_index]
self.current_index += 1
return next_word
else:
raise StopIteration
def __iter__(self):
self.current_index = 0
return self

hunk = WordHunk("These are some words")
for word in hunk:
print(word)

These
are
some
words

But wait, there’s more!

Defining both __iter__ and __next__ is a little tiresome, so Python provides some syntactic sugar to help us along.

We can use the magic of generators!

Esssentially, generators let us write a single method that implements the behavior of both __iter__ and __next__.

The syntax is pretty simple. We just need to write a normal method that will return values one at a time (like __next__ did), but instead of using a return statement, we’ll use yield instead. We can call this new method __iter__ and completely forget about __next__.

yield is pretty magical. It remembers exactly where it stopped returning values and always picks back up where it left off. It even takes care of throwing a StopIteration error for us.

Generators are a little complicated, but an example should clear things up.

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
def __str__(self):
return str(self.words)
def __repr__(self):
return "WordHunk('{}')".format(" ".join(self.words))
def __len__(self):
return len(self.words)
def __contains__(self, item):
return item in self.words
def __getitem__(self, key):
return self.words[key]
def __setitem__(self, key, new_value):
self.words[key] = new_value
def __iter__(self):
for word in self.words:
yield word

hunk = WordHunk("These are some words")
for word in hunk:
print(word)

These
are
some
words

Now that is a lot simpler!

Doing “Math” with WordHunks

What do you think would happen if we added two WordHunks together using the + operator?

I think that addition of two WordHunks should return a new WordHunk with the word lists concatenated together.

Something like this:

hunk = WordHunk("These are some words")
another_hunk = WordHunk("and they are great")
combined_hunk = hunk + another_hunk
combined_hunk

---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-29-e857aae79163> in <module>()
1 hunk = WordHunk("These are some words")
2 another_hunk = WordHunk("and they are great")
----> 3 combined_hunk = hunk + another_hunk
4 combined_hunk
TypeError: unsupported operand type(s) for +: 'WordHunk' and 'WordHunk'

Let’s make it happen!

To get addition to work, we need to define the __add__ method. It takes an argument (the thing that we want to add to our WordHunk) and returns a new WordHunk just like we talked about earlier.

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
def __str__(self):
return str(self.words)
def __repr__(self):
return "WordHunk('{}')".format(" ".join(self.words))
def __len__(self):
return len(self.words)
def __contains__(self, item):
return item in self.words
def __getitem__(self, key):
return self.words[key]
def __setitem__(self, key, new_value):
self.words[key] = new_value
def __iter__(self):
for word in self.words:
yield word
def __add__(self, other_WordHunk):
all_the_words_list = self.words + other_WordHunk.words
all_the_words_string = " ".join(all_the_words_list)
return WordHunk(all_the_words_string)

hunk = WordHunk("These are some words")
another_hunk = WordHunk("and they are great")
combined_hunk = hunk + another_hunk
combined_hunk

WordHunk('These are some words and they are great')

Fantastic!

Note: Since WordHunks are created from strings, we had to convert the new word list into a string.

Conclusion

And there you go!

Thanks to Python’s magic methods, our WordHunk is really featureful.

Check it out!

class WordHunk(object):
def __init__(self, word_string):
self.words = word_string.split()
def __str__(self):
return str(self.words)
def __repr__(self):
return "WordHunk('{}')".format(" ".join(self.words))
def __len__(self):
return len(self.words)
def __contains__(self, item):
return item in self.words
def __getitem__(self, key):
return self.words[key]
def __setitem__(self, key, new_value):
self.words[key] = new_value
def __iter__(self):
for word in self.words:
yield word
def __add__(self, other_WordHunk):
all_the_words_list = self.words + other_WordHunk.words
all_the_words_string = " ".join(all_the_words_list)
return WordHunk(all_the_words_string)

This list of magic methods is far from complete, but it should serve as a good introduction to the kinds of voodoo you have at your disposal.

If you’re hungry for more, I think that this guide is a great place to go next. It also isn’t complete, but it’ll give you enough magic methods to solve 98% of your problems.

That last 2% is really esoteric (for example, you can manipulate how Python stores your class’s attributes in memory). So I wouldn’t worry to much about it. When you need one of those crazy methods, you’ll know it.

Finally Learn Python Imports

Mon, 01 Jan 0001 00:00:00 +0000

What Is an Import?

Imports are statements that allow us to use code that lives somewhere different from where we’re currently working. We cannot (or rather, we absolutely should not) develop even a modest Python program without them. They’re essential.

They’re also significantly more complicated than most folks assume - filled with plenty of traps for unwary programmers to fall into.

In this notebook, we’ll cover imports from the basics (“What even IS an import?") to the edge of what is possible. (Note: we’re speaking from a Python3.2+ perspective).

Dig in, and enjoy!

Where Does Python Code Live?

Before we can start talking about importing Python code from various places, we need to first discuss the places in which Python code lives.

Modules

Python code lives in files that end in “.py”. In Python-speak, a file that ends in “.py” is also a module.

For example, let’s write some Python code to create a dinner plate and put it in a file named dinner_plate.py

dinner_plate.py

class DinnerPlate( object ):
def __repr__( self ):
return "DinnerPlate()"
def make_dinner_plate():
return DinnerPlate()

Here’s what our world looks like:

.
├── dinner_plate.py
└── GuideToImports.ipynb # We're right here!

We’re currently inside of the GuideToImports.ipynb Jupyter Notebook, and right beside us is a single file named dinner_plate.py. Since Python files are modules, we could also say that we have a single module named dinner_plate.

There’s a weird convention here that we need to talk about.

Notice that when we mentioned the file, we called it dinner_plate.py. But when we mentioned the module, we called it dinner_plate. There’s only one object here, but we use two different names for it. When we’re talking about this object as a file, we call it dinner_plate.py - with the “.py”. But when we’re talking about this object as a module, we call it dinner_plate - without the “.py”.

Yeah, yeah, yeah. It sounds totally stupid to have two slightly different names for the same thing. But, believe it or not, this is going to end up mattering a LOT.

Packages

If Python code lives in modules, then where do modules live?

Modules live in folders (also called directories), or in Python-speak, packages. Let me repeat that: packages are just folders.

Just as modules allow us to group Python snippets together, packages allow us to group modules together.

For example, let’s say that, in addition to our dinner_plate module, we create a similar module, cup, and we put it right beside dinner_plate.

cup

class Cup( object ):
def __repr__( self ):
return "Cup()"
def make_cup():
return Cup()

Here’s what our world looks like now:

.
├── cup.py
├── dinner_plate.py
└── GuideToImports.ipynb # We're right here!

Now this is all well and good, but we might want to group dinner_plate and cup. together into a package named flatware. So, we create a folder named flatware and move cup and dinner_plate into that folder.

Now we have a package named flatware that contains two modules named cup and dinner_plate.

.
├── flatware
│ ├── cup.py
│ └── dinner_plate.py
└── GuideToImports.ipynb # We're right here!

Lastly, packages can also have more packages inside of them! We call these inner packages subpackages, but there’s nothing special about them. They just regular packages that happen to live inside of another package.

For example, let’s create a subpackge inside of flatware named utensils and add two modules to it: fork and spoon.

fork.py

class Fork( object ):
def __repr__( self ):
return "Fork()"
def make_fork():
return Fork()

spoon.py

class Spoon( object ):
def __repr__( self ):
return "Spoon()"
def make_spoon():
return Spoon()

Now we have a package named flatware that contains two modules, cup and dinner_plate, as well as a subpacakge named utensilts. The utensils subpackage has two modules,fork and spoon.

.
├── flatware
│ ├── cup.py
│ ├── dinner_plate.py
│ └── utensils
│ ├── fork.py
│ └── spoon.py
└── GuideToImports.ipynb # We're right here!

Imports 101: The Basics of Using Imports

Now that we know where Python code lives, it’s time to get back to our original question.

How do we import a Python object from where it lives into where we want to use it?

To import an object, we need to do two things:

Identify the complete path from the top of our project to the object we want to import
Write an import statement

Keeping these two things in mind, let’s go import some stuff!

Importing a Module

Let’s try to import the fork module!

Step 1: Identifying the Complete Path

We need to get the complete path from the top of our project to where fork is.

.
├── flatware
│ ├── cup.py
│ ├── dinner_plate.py
│ └── utensils
│ ├── fork.py
│ └── spoon.py
└── GuideToImports.ipynb # We're right here!

To get from the top of our project to fork, we have to go into the flatware package, then into the utensils package, and finally into the fork module.

This means that the complete path from where we are to fork is:

flatware.utensils.fork

Step 2: Writing the Import Statement

Now that we have our path, flatware.utensils.fork, we’re ready to write a real Python import statement.

Here’s the syntax:

import flatware.utensils.fork

And that we have it! We have successfully imported the fork module!

Take note: import statements import modules, not files. This means that we imported the fork module, not the fork.py file.

We should never us “.py” when we’re trying to import a module. Because modules do not end in “.py”

(See! The whole file vs. module thing from earliler came in handy!)

Using the Imported Module

Now that we’ve imported fork, we’re now ready to use the two objects inside of fork: the class Fork and the function make_fork.

To use one of these objects - for example, the function make_fork - we need to use the complete path to make_fork: flatware.utensils.fork.make_fork

import flatware.utensils.fork
flatware.utensils.fork.make_fork()

Fork()

We got a fork object! Hurray!

That was pretty painless. The only headache was how much we had to type in order to call the make_fork function: flatware.utensils.fork.make_fork()

Luckily, Python provides a handy way around that.

Importing an Individual Object from a Module

In our previous example, we imported the fork module and then accessed the make_fork function inside of fork.

This time, we’re going to try something different. Instead of importing the fork module, we’re going to import an single object from the fork module.

For example, we can reach into the fork module and import just the make_fork function:

from flatware.utensils.fork import make_fork
make_fork()

Fork()

And there’s our Fork again!

Notice how we no longer have to call flatware.utensils.fork.make_fork(), we can simply write make_fork(). Convenient!

Take note: we didn’t import all of the objects that live in the fork module; we just imported make_fork. If we tried to access anything else from the fork module, we would run into an error.

Luckily, this “from … " syntax doesn’t limit us to importing only a single object. If we wanted to import the class Fork and the function make_fork from the fork module, we can do that in a single statement:

from flatware.utensils.fork import make_fork, Fork
make_fork(), Fork()

(Fork(), Fork())

Nicknaming Our Imported Objects

We’ve now seen two different kinds of imports:

import a.path.to.a.module
and from a.path.to.a.module import something

But wait! There’s more!

We can also give nicknames to the objects that we import!

Check this out:

import flatware.utensils.fork as my_fork_module
my_fork_module.make_fork()

Fork()

In writing as my_fork_module, we’re giving a nickname to my_fork_module that only applies here. The original fork module didn’t change.

Now when we want to call the function make_fork, we can use our nickname, my_fork_module.

Additionally, we can use nicknames with from imports!

from flatware.utensils.fork import make_fork as fork_function
fork_function()

Fork()

We can even import multiple objects and rename them:

from flatware.utensils.fork import make_fork as fork_function, Fork as ForkClass
fork_function(), ForkClass()

(Fork(), Fork())

Absolute and Relative Imports

We haven’t mentioned this before, but there are actually two different kinds of imports: absolute imports and relative imports.

Absolute Imports

With absolute imports, we have to include the name of every package and module in the path from the top of our project to the object we want to import.

absolute imports are what we’ve been using this whole time.

Relative Imports

Relative imports are different.

With relative imports, we don’t have to include the name of every pacakge in our path. Instead, we can use special nicknames that Python assigns to packages depending on their position relative to where we are.

Specifically:

the package we’re currently inside is nicknamed: . (a single dot)
the package one level above us is nicknamed: .. (two dots)
the package two levels above us is nicknamed: … (three dots)

… and so on…

As an example, let’s say we’re working inside of fork.py, and we want to import the make_cup function from cup.py and the make_spoon function from spoon.py.

.
├── flatware
│ ├── cup.py
│ ├── dinner_plate.py
│ └── utensils
│ ├── fork.py # Now we're right here!
│ └── spoon.py
└── GuideToImports.ipynb

If we were using absolute imports, we would the full path from the beginning of our project to the objects that we want to import:

from flatware.utensils.spoon import make_spoon
from flatware.cup import make_cup

But with relative imports, we can replace package names with dots.

Since we’re currently in fork.py…

. (single dot) refers to our current package: flatware.utensils
”.." (double dot) refers to the pacakge above us: flateware

Making those replacements yields this:

from .spoon import make_spoon
from ..cup import make_cup

Important Note: If we want to use relative imports, we must use from.

Absolute and Relative: Which Should We Use?

Generally speaking, we should prefer absolute imports over relative imports.

Absolute imports are clearer than their relative counterparts. With absolute imports, we have the full and complete paths to our imported objects. With relative imports, we have to mentally convert the dots into packages and hope that we got it right.

Of course, there is one big advantages to relative imports. Relative imports don’t depend on the exact names of our packages. If we change the names of our packages but do not change their structure, absolute imports will break, but relative imports will still succeed.

Is the extra flexibility of relative imports worth sacrificing the clarity of absolute imports? Perhaps not. These days, most text-editors and IDEs are capable of automatically renaming things for us – we don’t need the flexibility of relative imports.

Of course this is a decision everyone has to make on their own.

Imports 102: Importing a Module vs. Running a Module vs. Running a File

Now that we have explored the basics, it’s time to venture off the garden path. Here’s perhaps the most horrifying element of Python imports…

We can have perfectly good imports that break completely depending on how we use our modules.

For example, if we have a file, cool_code.py, that imports some things, those imports will behave differently depending on if we…

Import the cool_code module
Run the cool_code module
Run the cool_code.py file

(Also! Notice how, once again, we’re differentiating between the cool_code module and the cool_code.py file.)

This sounds… nasty, confusing, and disappointing.

Let’s try and make sense of this using an example. Here’s a simple project:

vehicles
├── engines
│ └── truck_engine.py
└── trucks
└── chevy.py

truck_engine

class TruckEngine( object ):
def start( self ):
return "VROOM!"

chevy

from vehicles.engines.truck_engine import TruckEngine
class Silverado( object ):
def __init__( self ):
self.engine = TruckEngine()
def start( self ):
return self.engine.start()

We have a vehicle package with two subpackages: engines and trucks. The engines subpackage has a truck_engine module with a TruckEngine class. The trucks subpackage has a chevy module with a Silverado class.

The Silverado class needs an engine, so it imports Truck engine from the top of the project using the path vehicles.engines.truck_engine.

Importing a Module

Let’s try importing the Silverado class and using it!

from vehicles.trucks.chevy import Silverado
myTruck = Silverado()
myTruck.start()

'VROOM!'

We imported our Silverado class from the chevy module and used it; everything worked perfectly.

No surprises here!

Running a Python File

Now let’s try running our chevy.py file. Our imports are good, and our Python is solid. Everything should work fine. Let’s give it a go.

To run a Python file, we need to open up our command line (also called the terminal) and enter the python command followed by the path through our filesystem to our chevy.py file.

Like this…

%%bash
python vehicles/trucks/chevy.py

Traceback (most recent call last):
File "vehicles/trucks/chevy.py", line 1, in <module>
from vehicles.engines.truck_engine import TruckEngine
ModuleNotFoundError: No module named 'vehicles'
---------------------------------------------------------------------------
CalledProcessError Traceback (most recent call last)
<ipython-input-24-7f6debbca43d> in <module>
----> 1 get_ipython().run_cell_magic('bash', '', 'python vehicles/trucks/chevy.py\n')
~/.local/share/virtualenvs/GuideToImports-e6Nz-2ie/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
2350 with self.builtin_trap:
2351 args = (magic_arg_s, cell)
-> 2352 result = fn(*args, **kwargs)
2353 return result
2354
~/.local/share/virtualenvs/GuideToImports-e6Nz-2ie/lib/python3.7/site-packages/IPython/core/magics/script.py in named_script_magic(line, cell)
140 else:
141 line = script
--> 142 return self.shebang(line, cell)
143
144 # write a basic docstring:
</home/cody/.local/share/virtualenvs/GuideToImports-e6Nz-2ie/lib/python3.7/site-packages/decorator.py:decorator-gen-110> in shebang(self, line, cell)
~/.local/share/virtualenvs/GuideToImports-e6Nz-2ie/lib/python3.7/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
185 # but it's overkill for just that one bit of state.
186 def magic_deco(arg):
--> 187 call = lambda f, *a, **k: f(*a, **k)
188
189 if callable(arg):
~/.local/share/virtualenvs/GuideToImports-e6Nz-2ie/lib/python3.7/site-packages/IPython/core/magics/script.py in shebang(self, line, cell)
243 sys.stderr.flush()
244 if args.raise_error and p.returncode!=0:
--> 245 raise CalledProcessError(p.returncode, cell, output=out, stderr=err)
246
247 def _run_script(self, p, cell, to_close):
CalledProcessError: Command 'b'python vehicles/trucks/chevy.py\n'' returned non-zero exit status 1.

Oh, the horror!

What went wrong? Let’s check out the error message at the very top and see if we can make any sense of this mess…

Traceback (most recent call last):
File "vehicles/trucks/chevy.py", line 1, in <module>
from vehicles.engines.truck_engine import TruckEngine
ModuleNotFoundError: No module named 'vehicles'

When our chevy module tried to import TruckEngine, it failed because it couldn’t find a module named vehicles.

This is confusing for at least two reasons:

This import worked just fine a minute ago
What does Python mean by No module named ‘vehicle’? Vehicle is a package! And it’s right there!

To understand what’s going on, we need to understand how Python actually finds modules to import.

How Python Finds Modules

When Python starts up, it creates a giant list of places where it’ll look for modules that we want to import. If our module isn’t in one of those places on the list, Python won’t find it and all sorts of nasty errors will pop up.

This giant list of places is actually just a variable named path that lives inside of Python’s built-in sys module. It’s just a regular Python list with regular Python strings inside of it. We can import it and muck around with it just like any other list.

Weirdly enough, the list sys.path starts out completely empty. There’s nothing there, and if it stayed that way, we wouldn’t be able to import anything. Luckily, Python automatically adds some things to sys.path.

Python adds the directory containing all of Python’s built-in modules
Python adds the directory containing all of the extra packages we’ve personally installed
And if we run a Python file (like we did just a minute ago), Python adds the directory containing the file we ran.

Yeah, yeah, yeah. This is super interesting or whatever, BUT WHAT DOES ANY OF THIS MEAN???

Our error noted that Python couldn’t find the vehicles package, so vehicles must not have been in sys.path!

So let’s think about what was in sys.path. As per above, sys.path has the directory containing all of the built-in modules, the directory containing all of our installed modules, and finally, sys.path has the directory containing the file we ran.

Since we ran python vehicles/trucks/chevy.py, the directory containing chevy.py was trucks/. So, the trucks/ direcgtory got added in sys.path.

See the problem? The vehicles/ directory never got added to sys.path! This means that our import statement inside of the chevy module…

from vehicles.engines.truck_engine import TruckEngine

…completely fails because Python cannot find the vehicles/ part of our path!

So… Can We Fix This? Or Can We Never Run Python Code Again?

Well, there’s the naive solution: right before Python reaches our from vehicles.engines.truck_engine ... import statement, we could manually add the vehicles/ directory to sys.path. After all, sys.path is just a list. We can add things to it.

This would work… but it is a horrible solution. It it just so hacky, so adhoc, and so ugly. It is also annoying: we don’t want to add this…

import sys.path
sys.path.append( "some/path/to/some/place" )`

…to the top of every file that might have a problematic import. That’s terrible.

Luckily, There Is Another Way!

Instead of running the file vehicles/trucks/chevy.py, we can instead run the module vehicles.trucks.chevy

If we run the module chevy, Python does something slightly different to sys.path than when we ran the file chevy.py. Instead of adding the directory containing the file (which was /trucks), Python will add our current directory to sys.path.

Let’s remind ourselves what our current directory is…

. # <--- This is our current directory
├── GuideToImports.ipynb # We're right here!
├── flatware
│ ├── cup.py
│ ├── dinner_plate.py
│ └── utensils
│ ├── fork.py
│ └── spoon.py
└── vehicles
├── engines
│ └── truck_engine.py
└── trucks
└── chevy.py

Perfect! Our current directory contains vehicles/

This means that, as long as we run the chevy module instead of the chevy.py file, the vehicles package will be added to sys.path, thus allowing Python to find vehicles and everything underneath it!

But How Do We Run Something as a Module?

As it turns out, it’s trivially easy to run something as a module instead of as a file.

Instead of running…

%%bash
python vehicles/trucks/chevy.py

…in our command line / terminal, we need to run…

%%bash
python -m vehicles.trucks.chevy

AND IT WORKS! WE DON’T GET AN ERROR!

There are two things to notice:

We added “-m” to our command. This tells Python that we’re running a module and not a file.
We used “vehicles.trucks.chevy”, not “vehicles/trucks/chevy.py”. This is because…
- Files are separated by slashes, but packages and modules (like all of our imports) are separated by dots.
- Pyton files end in “.py”, but modules do not

Once again, the seemingly trivial difference between files and modules saves the day once again.

This Has Been a Long, Difficult Section. Let’s Summarize.

If we want to import modules, there’s no need to worry. As long as we wrote valid import statements, Python can figure everything out for us.
But if we want to run a file or a module, we need to think about sys.path
- No matter what, sys.path will contain the directories where built-ins and user-installed modules live.
- If we run a Python file, sys.path will contain the directory where our file lives
- If we run a Python module, sys.path will contain our current directory.
  - You run a Python file like this: python path/to/your/file.py
  - And you run a Python module like this: python path.to.your.file

Imports 201: Best Practices

Now that we understand how to use imports, it’s time to understand how to use imports properly.

Where Do We Put Our Imports?

We put imports at the very top of our Python modules, right after any comments/docstrings, and right before any module-level variables.

"""
Here are some comments! Comment, comment, comment!
"""
import math
import sys
MODULE_LEVEL_VARIABLE = ( "Big", "Old", "Variable" )

Every other option is terrible. Full stop. Do not suprise folks with hidden imports.

How Do We Organize Our Imports?

Every new import goes on a new line. The only exception are from... imports.

import math
import sys
from my_module import function_one, function_two

Group imports from the same packages together

import math
import my_package.my_module # Imports from `my_package` are together
import my_package.my_other_module

You should have three different sections of imports, each separated by a new line. Those sections are…
- Built-in imports
- Third-party imports
- Local imports

import math #
import os # math and os are built-in
import pandas
import requests # pandas and requests are third-party modules
from ..my_package.my_module import my_function
import my_pacakge.my_subpackage.my_other_module # These are local imports

How Do We Feel About Relative vs. Absolute Imports Again?

We generally prefer absolute imports over relative imports.
- Absolute imports make it clearer where the import is coming from.
  - But if your absolute imports are getting incredibly long, relative imports can help.
  - Additionally, if you think that the names of your packages will change (but their structure won’t), relative imports can also help.

from my_package.my_subpackage.my_module import my_function # We tend to prefer this...
from ..my_subpackage.my_module import my_function # ...over this.

How Do We Feel About Nicknames?

We treat them with suspicion, especially if we’re nicknaming individual functions, classes, and variables.

import my_package.my_subpackage.my_subsubpackage.my_module as my_module # This is...fine.
import my_module as jelly_beans # This is seriously questionable
from my_module import function1 as function2 # This is a horrible idea

How Do We Feel About Using “from…” to Import Subpackages?

We feel okay about importing subpackages, as long as you make sure that your subpackage doesn’t share the same name as a third-party or built-in object.

from my_package import my_subpackage # We feel okay about this...
#...if my_subpackage is unique.

Can We Import Something From Somewhere Other Than Where It Was Defined?

Ugh… Yeah.. We can do this. BUT We SHOULDN’T! We should import objects from the module in which they were originally defined.

module_a

from my_package.my_subpackage import my_function

module_b

from module_a import my_function # BOO! HISS! TERRIBLE!

Where to Go from Here?

Onwards into the Python documentation!

tutorial | Cody VanZandt

Better Coding Through Shakespeare

Better Coding Through Shakespeare

Act I: Naming Things. Or, Romeo Is a Dreadful Programmer

See Names as Opportunities to Add Value

i,j,k (and Friends) Are Only For Indices

Don’t Add Types If They Don’t Add Information

Make Sure You Can Pronounce It

Variables and Classes are Nouns. Functions and Methods are Verbs.

Pick Your Conventions and Stick with Them

Act II: Functions. Or, Hermia Would Make a Good Function

Function Trick #1: Functions Should Be Small

Function Trick #2: Functions Should Do One Thing

Function Trick #3: Functions Should Operate at a Single Level of Abstraction

Act III: “Come hither, come hither. How did this argument begin?” Or, Arguments Confuse Don Adriano (And Everyone Else)

Argument Trick #1: Make Your Arguments Classy

Argument Trick #2: Make Sure Arguments Don’t Come OUT of Functions

Argument Trick #3: Flags are for Countries, Not for Programmers

Argument Trick #4: Leave “None” Alone!

Act IV: “Forgive the comment that my passion made / Upon thy feature” [11] Or, King John Regrets Commenting and You Should Too

Dire Warnings

Explaining Interfaces You Don’t Own

Explaining the Truly Esoteric

Licenses, Test Files, Permissions, Primary Maintainers, and Other Necessities

Act V: “You are too blunt. Go to it orderly.” Or, Petruchio Needs Better Formatting

How Long Should a File Be?

Where Should I Put Things?

Do Not Line Up Your Assignments

Act VI: “Doth not the object cheer your heart, my lord?” Or, Queen Margaret Loves Object-Oriented Programming

When Should You Use a Class?

How Big Should Classes Be?

Act VII: Final Words, Or What You Will

Works Cited

The Sorcerer's Code

We’re Going to Need a Little Magic

Magic Methods!

The __init__ Method

The __str__ Method

The __repr__ Method

The __len__ Method

The __contains__ Method

The __getitem__ Method

The __setitem__ Method

The __iter__ Method

Doing “Math” with WordHunks

Conclusion

Finally Learn Python Imports

What Is an Import?

Where Does Python Code Live?

Modules

Packages

Imports 101: The Basics of Using Imports

Importing a Module

Step 1: Identifying the Complete Path

Step 2: Writing the Import Statement

Using the Imported Module

Importing an Individual Object from a Module

Nicknaming Our Imported Objects

Absolute and Relative Imports

Absolute Imports

Relative Imports

Absolute and Relative: Which Should We Use?

Imports 102: Importing a Module vs. Running a Module vs. Running a File

Importing a Module

Running a Python File

How Python Finds Modules

So… Can We Fix This? Or Can We Never Run Python Code Again?

Luckily, There Is Another Way!

But How Do We Run Something as a Module?

This Has Been a Long, Difficult Section. Let’s Summarize.

Imports 201: Best Practices

Where Do We Put Our Imports?

How Do We Organize Our Imports?

How Do We Feel About Relative vs. Absolute Imports Again?

How Do We Feel About Nicknames?

How Do We Feel About Using “from…” to Import Subpackages?

Can We Import Something From Somewhere Other Than Where It Was Defined?

Where to Go from Here?

The `init` Method

The `str` Method

The `repr` Method

The `len` Method

The `contains` Method

The `getitem` Method

The `setitem` Method

The `iter` Method