## Serious Python  ## Metadata - Author: - Full Title: Serious Python - Category: #books ## Highlights ### 5 DISTRIBUTING YOUR SOFTWARE - The packaging ecosystem in Python has a bumpy history; however, the situation is now settling. ### 3 DOCUMENTATION AND GOOD API PRACTICE - Having such examples in your documentation lets users understand your API. However, it’s easy to put off and eventually forget to update your examples as your API evolves. Fortunately, doctest helps make sure this doesn’t happen. If your documentation includes a step-by-step tutorial, doctest will help you keep it up to date throughout development by testing every line it can. You can also use doctest for documentation-driven development (DDD): write your documentation and examples first and then write code to match your documentation. Taking advantage of this feature is as simple as running sphinx-build with the special doctest builder, like this: - The creators of Web Services Made Easy (WSME) (interviewed at the end of this chapter) have come up with a solution: a Sphinx extension called sphinxcontrib-pecanwsme that analyzes docstrings and actual Python code to generate REST API documentation automatically. - As another example, in one of my projects named Gnocchi—a database for storing and indexing time series data at a large scale—I’ve used a custom Sphinx extension to autogenerate documentation. Gnocchi provides a REST API, and usually to document such an API, projects will manually write examples of what an API request and its response should look like. Unfortunately, this approach is error prone and out of sync with reality. Using the unit-testing code available to test the Gnocchi API, we built a Sphinx extension to run Gnocchi and generate an .rst file containing HTTP requests and responses run against a real Gnocchi server. In this way, we ensure the documentation is up to date: the server responses are not manually crafted, and if a manually written request fails, then the documentation process fails, and we know that we must fix the documentation. Including that code in the book would be too verbose, but you can check the sources of Gnocchi online and look at the gnocchi.gendoc module to get an idea of how it works. - Sphinx is the de facto standard for documenting Python projects. It supports a wide variety of syntax, and it is easy to add new syntax or features if your project has particular needs. Sphinx can also automate tasks such as generating indexes or extracting documentation from your code, making it easy to maintain documentation in the long run. Documenting changes to your API is critical, especially when you deprecate functionality, so that users are not caught unawares. Ways to document deprecations include the Sphinx deprecated keyword and the warnings module, and the debtcollector library can automate maintaining this documentation. - Write unit tests. TDD (test-driven development) is a very efficient way to write libraries, especially in Python, because it forces the developer to assume the role of the end user from the very beginning, which leads the developer to design for usability. It’s the only approach I know of that allows a programmer to completely rewrite a library, as a last resort. - Document early and include your documentation build in continuous integration. With the Read the Docs tool for creating and hosting documentation, there’s no excuse for not having documentation built and published (at least for open source software). - Give practical examples throughout. Have at least one “startup guide” that will show newcomers how to build a working example. The first page of the documentation should give a quick overview of your API’s basic and representative use case. ### 8 FUNCTIONAL PROGRAMMING - When you write code using a functional style, your functions are designed to have no side effects: instead, they take an input and produce an output without keeping state or modifying anything not reflected in the return value. Functions that follow this ideal are referred to as purely functional. - Modularity Writing with a functional style forces a certain degree of separation in solving your individual problems and makes sections of code easier to reuse in other contexts. Since the function does not depend on any external variable or state, calling it from a different piece of code is straightforward. - Testability Testing a functional program is incredibly easy: all you need is a set of inputs and an expected set of outputs. They are idempotent, meaning that calling the same function over and over with the same arguments will always return the same result. - Say we want to find the first number between 1 and 10,000,000 that’s equal to 50,000. Sounds easy, doesn’t it? Let’s make this a challenge. We’ll run Python with a memory constraint of 128MB and try the naive approach of first building the entire list: ### 9 THE ABSTRACT SYNTAX TREE, HY, AND LISP-LIKE ATTRIBUTES - The abstract syntax tree (AST) is a representation of the structure of the source code of any programming language. Every language, including Python, has a specific AST; Python’s AST is built by parsing a Python source file. Like any tree, this one is made of nodes linked together. A node can represent an operation, a statement, an expression, or even a module. Each node can contain references to other nodes that make up the tree. - The ast.parse() function parses any string that contains Python code and returns an _ast.Module object. That object is actually the root of the tree: you can browse it to discover every node making up the tree. To visualize what the tree looks like, you can use the ast.dump() function, which will return a string representation of the whole tree. - Tags: [[favorite]] - This whole process is exactly what happens when you run Python on a .py file: once the text tokens are parsed, they are converted into a tree of ast objects, compiled, and evaluated. - The arguments lineno and col_offset represent the line number and column offset, respectively, of the source code that has been used to generate the AST. It doesn’t make much sense to set these values in this context since we are not parsing a source file, but it can be useful to be able to find the position of the code that generated the AST. For example, Python uses this information when generating backtraces. - The functools.partial() method is typically useful in place of lambda and should be considered a superior alternative. The lambda function is something of an anomaly in the Python language, and dropping it altogether was considered for Python 3 due to the function’s limited body size of a single line. - Using functional programming as a paradigm in Python can help you make your program more reusable and easier to test and debug, supporting the Don’t Repeat Yourself (DRY) mantra. In this spirit, the standard Python modules itertools and operator are good tools to improve the readability of your functional code.