Integration of Naming Conventions into Operations, Autogenerate

it allows each new key to be given a default value based on the type of dictionary being created

Memoization can be achieved through Python decorators

import timeit

def memoize_fib(func):
    cache = {}
    def inner(arg):
        if arg not in cache:            
            cache[arg] = func(arg)
        return cache[arg]
    return inner


def fib(num):
    if num == 0:
        return 0
    elif num == 1:
        return 1
    else:
        return fib(num-1) + fib(num-2)

fib = memoize_fib(fib)


print(timeit.timeit('fib(30)', globals=globals(), number=1))
print(timeit.timeit('fib(30)', globals=globals(), number=1))
print(timeit.timeit('fib(30)', globals=globals(), number=1))

4.9035000301955733e-05
1.374000021314714e-06
1.2790005712304264e-06

A deep copy refers to cloning an object. When we use the = operator, we are not cloning the object; instead, we reference our variable to the same object (a.k.a. shallow copy).

Python 2 is entrenched in the software landscape to the point that co-dependency between several softwares makes it almost impossible to make the shift.

The @property decorator allows a function to be accessed like an attribute.

var = true_val if condition else false_val

to_check = 6
msg = "Even" if to_check%2 == 0 else "Odd"
print(msg)

msg = ""
if(to_check%2 == 0):
  msg = "Even"
else:
  msg = "Odd"
print(msg)

This method is automatically called to allocate memory when a new object/ instance of a class is created.

decorator is a design pattern in Python that allows a user to add new functionality to an existing object without modifying its structure

The philosophy behind Flask is that it gives only the components you need to build an app so that you have the flexibility and control. In other words, it’s un-opinionated. Some features it offers are a build-int dev server, Restful request dispatching, Http request handling, and much more.

The only difference is that range returns a Python list object and xrange returns an xrange object. This means that xrange doesn’t actually generate a static list at run-time like range does.

NumPy is not just more efficient; it is also more convenient. You get a lot of vector and matrix operations for free, which sometimes allow one to avoid unnecessary work. And they are also efficiently implemented.

process of retrieving original Python objects from the stored string representation

Pickle module accepts any Python object and converts it into a string representation and dumps it into a file by using dump function

The beauty of lambda functions lies in the fact that they return function objects. This makes them helpful when used with functions like map or filter which require function objects as arguments.

In Python, the term monkey patch only refers to dynamic modifications of a class or module at run-time.

Python has a multi-threading package but if you want to multi-thread to speed your code up, then it’s usually not a good idea to use it. Python has a construct called the Global Interpreter Lock (GIL). The GIL makes sure that only one of your ‘threads’ can execute at any one time. A thread acquires the GIL, does a little work, then passes the GIL onto the next thread.

content Gray Hat Python

Hot Reloading refers to the ability to automatically update a running web application when changes are made to the application’s code.

JupyterDash supports three approaches to displaying a Dash application during interactive development.

# Run app and display result inline in the notebookapp.run_server(mode='inline')

app.run_server(mode='inline')

If running the server blocks the main thread, then it’s not possible to execute additional code cells without manually interrupting the execution of the kernel.JupyterDash resolves this problem by executing the Flask development server in a background thread. This leaves the main execution thread available for additional calculations. When a request is made to serve a new version of the application on the same port, the currently running application is automatically shut down first. This makes is possible to quickly update a running application by simply re-executing the notebook cells that define it.

You can also try it out, right in your browser, with binder.

Then, copy any Dash example into a Jupyter notebook cell and replace the dash.Dash class with the jupyter_dash.JupyterDash class.

from jupyter_dash import JupyterDash

import dash

import plotly.express as px
from jupyter_dash import JupyterDash
import dash_core_components as dcc
import dash_html_components as html
from dash.dependencies import Input, Output

there is a particularly unconventional mechanism by which these coroutines actually get run. Their result is an attribute of the exception object that gets thrown when their .send() method is called.

If Python encounters an await f() expression in the scope of g(), this is how await tells the event loop, “Suspend execution of g() until whatever I’m waiting on—the result of f()—is returned. In the meantime, go let something else run.”

A coroutine is a specialized version of a Python generator function

Asynchronous routines are able to “pause” while waiting on their ultimate result and let other routines run in the meantime. Asynchronous code, through the mechanism above, facilitates concurrent execution. To put it differently, asynchronous code gives the look and feel of concurrency.

Python 中的变量赋值不需要类型声明。 每个变量在内存中创建，都包括变量的标识，名称和数据这些信息。 每个变量在使用前都必须赋值，变量赋值以后该变量才会被创建。 等号（=）用来给变量赋值。 等号（=）运算符左边是一个变量名,等号（=）运算符右边是存储在变量中的值。

像if、while、def和class这样的复合语句，首行以关键字开始，以冒号( : )结束，该行之后的一行或多行代码构成代码组。 我们将首行及后面的代码组称为一个子句(clause)。

Python语句中一般以新行作为语句的结束符。 但是我们可以使用斜杠（ \）将一行的语句分为多行显示，如下所示：

在 Python 里，标识符由字母、数字、下划线组成。 在 Python 中，所有标识符可以包括英文、数字以及下划线(_)，但不能以数字开头。 Python 中的标识符是区分大小写的。 以下划线开头的标识符是有特殊意义的。以单下划线开头 _foo 的代表不能直接访问的类属性，需通过类提供的接口进行访问，不能用 from xxx import * 而导入。 以双下划线开头的 __foo 代表类的私有成员，以双下划线开头和结尾的 __foo__ 代表 Python 里特殊方法专用的标识，如 __init__() 代表类的构造函数

python -m http.server 8000

You can create estimation plots here at estimationstats.com, or with the DABEST packages which are available in R, Python, and Matlab.

Kolejna zmienna - "Recency", czyli informacja o tym, jak dawno klient robił zakupy w sklepie.

df['Recency'] = (today - df.InvoiceDate)/np.timedelta64(1,'D')

abt = df.groupby(['CustomerID']).agg({'Recency':'min', 'MonetaryValue':'sum', 'InvoiceNo':'count'})

abt = df.groupby(['CustomerID']).agg({'Recency':'min', 'MonetaryValue':'sum', 'InvoiceNo':'count'})
abt.rename(columns = {'InvoiceNo':'Frequency'}, inplace = True)
abt = abt[['Recency', 'Frequency', 'MonetaryValue']]
abt.head()

Usuwam brakujące wartości w zmiennej "CustomerID".

df = df[~df.CustomerID.isnull()]

df['CustomerID'] = df.CustomerID.astype(int)

df.drop(['Description', 'StockCode', 'Country'], axis = 1, inplace = True)

Sprawdzam braki danych.

print(str(round(df.isnull().any(axis=1).sum()/df.shape[0]*100,2))+'% obserwacji zawiera braki w danych.')

24.93% obserwacji zawiera braki w danych.

Buduję prosty data frame z podstawowymi informacjami o zbiorze.

summary = pd.DataFrame(df.dtypes, columns=['Dtype'])
summary['Nulls'] = pd.DataFrame(df.isnull().any())
summary['Sum_of_nulls'] = pd.DataFrame(df.isnull().sum())
summary['Per_of_nulls'] = round((df.apply(pd.isnull).mean()*100),2)
summary.Dtype = summary.Dtype.astype(str)
print(summary)

                      Dtype  Nulls  Sum_of_nulls  Per_of_nulls
InvoiceNo            object  False             0         0.000
StockCode            object  False             0         0.000
Description          object   True          1454         0.270
Quantity              int64  False             0         0.000
InvoiceDate  datetime64[ns]  False             0         0.000
UnitPrice           float64  False             0         0.000
CustomerID          float64   True        135080        24.930
Country              object  False             0         0.000

Programming languages These will probably expose my ignorance pretty nicely.

Multi-level argparse in Python (parsing commands like git)

Python String Formatting Best Practices

In Python, when trying to do a dubious operation, you get an error pretty soon. In JavaScript… an undefined can fly through a few layers of abstraction, causing an error in a seemingly unrelated piece of code.

function add(a,b) { return + (a + b) }
add(2,2)
add('2', 2)

How to use the PPS in your own (Python) project

Sometimes, the best way to learn is to mimic others. Here are some great examples of projects that use documentation well:

Class method docstrings should contain the following: A brief description of what the method is and what it’s used for Any arguments (both required and optional) that are passed including keyword arguments Label any arguments that are considered optional or have a default value Any side effects that occur when executing the method Any exceptions that are raised Any restrictions on when the method can be called

From examining the type hinting, you can immediately tell that the function expects the input name to be of a type str, or string. You can also tell that the expected output of the function will be of a type str, or string, as well.

def hello_name(name: str) -> str:
    return(f"Hello {name}")

Docstrings can be further broken up into three major categories: Class Docstrings: Class and class methods Package and Module Docstrings: Package, modules, and functions Script Docstrings: Script and functions

Docstring conventions are described within PEP 257. Their purpose is to provide your users with a brief overview of the object.

All multi-lined docstrings have the following parts: A one-line summary line A blank line proceeding the summary Any further elaboration for the docstring Another blank line

"""This is the summary line

This is the further elaboration of the docstring. Within this section,
you can elaborate further on details as appropriate for the situation.
Notice that the summary and the elaboration is separated by a blank new
line.

# Notice the blank line above. Code should continue on this line.

say_hello.__doc__ = "A simple function that says hello... Richie style"

def say_hello(name):
    print(f"Hello {name}, is it me you're looking for?")

say_hello.__doc__ = "A simple function that says hello... Richie style"

def say_hello(name):
    """A simple function that says hello... Richie style"""
    print(f"Hello {name}, is it me you're looking for?")

>>> help(say_hello)

Help on function say_hello in module __main__:

say_hello(name)
    A simple function that says hello... Richie style

class constructor parameters should be documented within the __init__ class method docstring

Scripts are considered to be single file executables run from the console. Docstrings for scripts are placed at the top of the file and should be documented well enough for users to be able to have a sufficient understanding of how to use the script.

Documenting your code, especially large projects, can be daunting. Thankfully there are some tools out and references to get you started

Along with these tools, there are some additional tutorials, videos, and articles that can be useful when you are documenting your project

If you use argparse, then you can omit parameter-specific documentation, assuming it’s correctly been documented within the help parameter of the argparser.parser.add_argument function. It is recommended to use the __doc__ for the description parameter within argparse.ArgumentParser’s constructor.

There are specific docstrings formats that can be used to help docstring parsers and users have a familiar and known format.

Daniele Procida gave a wonderful PyCon 2017 talk and subsequent blog post about documenting Python projects. He mentions that all projects should have the following four major sections to help you focus your work:

Since everything in Python is an object, you can examine the directory of the object using the dir() command

Documenting your Python code is all centered on docstrings. These are built-in strings that, when configured correctly, can help your users and yourself with your project’s documentation.

Along with docstrings, Python also has the built-in function help() that prints out the objects docstring to the console.

The general layout of the project and its documentation should be as follows:

project_root/
│
├── project/  # Project source code
├── docs/
├── README
├── HOW_TO_CONTRIBUTE
├── CODE_OF_CONDUCT
├── examples.py

In all cases, the docstrings should use the triple-double quote (""") string format.

Since we have much faster CPUs now, numerical calculations are done in Python which is much slower than Fortran. So numerical calculations basically take the same amount of time as they did 20 years ago.

Python unit testing libraries, like unittest, can be used within a notebook, but standard CI/CD tooling has trouble dealing with notebooks for the same reasons that notebook diffs are hard to read.

def handle_exception(self, job, *exc_info):

worker = Worker(..., exception_handler=[handle_exception])
try:
    raise Exception()
except Exception:
    exc_info = sys.exc_info()

worker.handle_exception(job, *exc_info)

Python contributed examples¶ Mic VAD Streaming¶ This example demonstrates getting audio from microphone, running Voice-Activity-Detection and then outputting text. Full source code available on https://github.com/mozilla/DeepSpeech-examples. VAD Transcriber¶ This example demonstrates VAD-based transcription with both console and graphical interface. Full source code available on https://github.com/mozilla/DeepSpeech-examples.

Python API Usage example Edit on GitHub Python API Usage example¶ Examples are from native_client/python/client.cc. Creating a model instance and loading model¶ 115 ds = Model(args.model) Performing inference¶ 149 150 151 152 153 154 if args.extended: print(metadata_to_string(ds.sttWithMetadata(audio, 1).transcripts[0])) elif args.json: print(metadata_json_output(ds.sttWithMetadata(audio, 3))) else: print(ds.stt(audio)) Full source code

failure_ttl

result_ttl=600

job.meta['handled_by'] = socket.gethostname() job.save_meta()

w = Worker([q], exception_handlers=[foo_handler, bar_handler])

def my_handler(job, exc_type, exc_value, traceback): # do custom things here

How speech recognition works, What packages are available on PyPI; and How to install and use the SpeechRecognition package—a full-featured and easy-to-use Python speech recognition library.

The Ultimate Guide To Speech Recognition With Python

How to setup and use Stanford CoreNLP Server with Python Khalid Alnajjar August 20, 2017 Natural Language Processing (NLP) Leave a CommentStanford CoreNLP is a great Natural Language Processing (NLP) tool for analysing text. Given a paragraph, CoreNLP splits it into sentences then analyses it to return the base forms of words in the sentences, their dependencies, parts of speech, named entities and many more. Stanford CoreNLP not only supports English but also other 5 languages: Arabic, Chinese, French, German and Spanish. To try out Stanford CoreNLP, click here.Stanford CoreNLP is implemented in Java. In some cases (e.g. your main code-base is written in different language or you simply do not feel like coding in Java), you can setup a Stanford CoreNLP Server and, then, access it through an API. In this post, I will show how to setup a Stanford CoreNLP Server locally and access it using python.

For example, the test_decrement functions given earlier are failing because the assertion itself is faulty.

Support for running tests in parallel with pytest is available through the pytest-xdist package.

With pytest, failed tests also appear in the Problems panel, where you can double-click on an issue to navigate directly to the test

VS Code also shows test results in the Python Test Log output panel (use the View > Output menu command to show the Output panel, then select Python Test Log

For Python, test discovery also activates the Test Explorer with an icon on the VS Code activity bar. The Test Explorer helps you visualize, navigate, and run tests

Once VS Code recognizes tests, it provides several ways to run those tests

You can trigger test discovery at any time using the Python: Discover Tests command.

Sometimes tests placed in subfolders aren't discovered because such test files cannot be imported. To make them importable, create an empty file named __init__.py in that folder.

Testing in Python is disabled by default. To enable testing, use the Python: Configure Tests command on the Command Palette.

python.testing.pytestArgs: Looks for any Python (.py) file whose name begins with "test_" or ends with "_test", located anywhere within the current folder and all subfolders.

python.testing.unittestArgs: Looks for any Python (.py) file with "test" in the name in the top-level project folder.

Create a file named test_unittest.py that contains a test class with two test methods

import inc_dec    # The code to test
import unittest   # The test framework

class Test_TestIncrementDecrement(unittest.TestCase):
    def test_increment(self):
        self.assertEqual(inc_dec.increment(3), 4) # checks if the results is 4 when x = 3

    def test_decrement(self):
        self.assertEqual(inc_dec.decrement(3), 4)

if __name__ == '__main__':
    unittest.main()

Nose2, the successor to Nose, is just unittest with plugins

Python tests are Python classes that reside in separate files from the code being tested.

general background on unit testing, see Unit Testing on Wikipedia. For a variety of useful unit test examples, see https://github.com/gwtw/py-sorting

each test is very simple: invoke the function with an argument and assert the expected return value.

    def test_validator_valid_string():
        # The exact assertion call depends on the framework as well
        assert(validate_account_number_format("1234567890"), true)

Visual Studio Code supports working with Jupyter Notebooks natively, as well as through Python code files.

# %%
msg = "Hello World"
print(msg)

# %%
msg = "Hello again"
print(msg)

JupyterLab project, which enables a richer UI including a file browser, text editors, consoles, notebooks, and a rich layout system.

Some of the most common operators used in filter() method SQLAlchemy

pip is a package manager. conda is both a package manager and an environment maanger.

Validators, like all attribute extensions, are only called by normal userland code; they are not issued when the ORM is populating the object

validates works only during append operation. Not during assignment

Note: When you create a new virtual environment, you should be prompted by VS Code to set it as the default for your workspace folder. If selected, the environment will automatically be activated when you open a new terminal.

Tip: Use Logpoints instead of print statements: Developers often litter source code with print statements to quickly inspect variables without necessarily stepping through each line of code in a debugger. In VS Code, you can instead use Logpoints. A Logpoint is like a breakpoint except that it logs a message to the console and doesn't stop the program. For more information, see Logpoints in the main VS Code debugging article.

Open an Anaconda command prompt and run conda create -n myenv python=3.7 pandas jupyter seaborn scikit-learn keras tensorflow

E713 test for membership should be ‘not in’

# Add auto-completion and a stored history file of commands to your Python # interactive interpreter. Requires Python 2.0+, readline. Autocomplete is # bound to the Esc key by default (you can change it - see readline docs). # # Store the file in ~/.pystartup, and set an environment variable to point # to it: "export PYTHONSTARTUP=~/.pystartup" in bash. import atexit import os import readline import rlcompleter historyPath = os.path.expanduser("~/.pyhistory") def save_history(historyPath=historyPath): import readline readline.write_history_file(historyPath) if os.path.exists(historyPath): readline.read_history_file(historyPath) atexit.register(save_history) del os, atexit, readline, rlcompleter, save_history, historyPath

Jupytext can be configured to automatically pair a git-friendly file for input data while preserving the output data in the .ipynb file

He discussed the history of Functional Programming language features in one of his blog posts. As a result, the language implementations have not been optimized for Functional Programming features.

Every map and filter expression can be expressed as a list comprehension.

Ever had a bug where you wondered how a variable you set to 25 became None? If that variable was immutable, the error would have been thrown where the variable was being changed, not where the changed value already affected the software - the root cause of the bug can be found earlier.

This denotes the factorial of a number. It is the product of numbers starting from 1 to that number.

x = 5
fact = 1
for i in range(x, 0, -1):
    fact = fact * i
print(fact)

import math
math.factorial(x)

# 5*4*3*2*1
120

The hat gives the unit vector. This means dividing each component in a vector by it’s length(norm).

x = [1, 2, 3]
length = math.sqrt(sum([e**2 for e in x]))
x_hat = [e/length for e in x]

math.sqrt(sum([e**2 for e in x_hat]))
# 1.0

It gives the sum of the products of the corresponding entries of the two sequences of numbers.

X = [1, 2, 3]
Y = [4, 5, 6]
dot = sum([i*j for i, j in zip(X, Y)])
# 1*4 + 2*5 + 3*6
# 32

It means multiplying the corresponding elements in two tensors. In Python, this would be equivalent to multiplying the corresponding elements in two lists.

x = [[1, 2], 
    [3, 4]]
y = [[2, 2], 
    [2, 2]]
z = np.multiply(x, y)

[[2, 4]],
[[6, 8]]

This is basically exchanging the rows and columns.

X = [[1, 2, 3], 
    [4, 5, 6]]
np.transpose(X)

[[1, 4], 
 [2, 5],
 [3, 6]]

This denotes a function which takes a domain X and maps it to range Y. In Python, it’s equivalent to taking a pool of values X, doing some operation on it to calculate pool of values Y.

def f(X):
    Y = ...
    return Y

This symbol checks if an element is part of a set.

X = {1,2,3}
3 in X

The norm is used to calculate the magnitude of a vector. In Python, this means squaring each element of an array, summing them and then taking the square root.

x = [1, 2, 3]
math.sqrt(x[0]**2 + x[1]**2 + x[2]**2)

This symbol denotes the absolute value of a number i.e. without a sign.

x = 10
y = -20
abs(x)  #10
abs(y) #20

In Python, it is equivalent to looping over a vector from index 0 to index N-1 and multiplying them.

we reuse the sigma notation and divide by the number of elements to get an average.

x = [1, 2, 3, 4, 5]
result = 0
N = len(x)
for i in range(n):
 result = result + x[i]
average = result / N
print(average)

x = [1, 2, 3, 4, 5]
result = sum(x) / len(x)

In Python, it is equivalent to looping over a vector from index 0 to index N-1

x = [1, 2, 3, 4, 5]
result = 0
N = len(x)
for i in range(N):
 result = result + x[i]
print(result)

x = [1, 2, 3, 4, 5]
result = sum(x)

2D vectors

x = [ [10, 20, 30], [40, 50, 60] ]
i = 0
j = 1
print(x[i][j]) # 20

GetDihedralConfDegree

form["username"]

python -m http.server --cgi 8000

Python is used, however, in researching new drug molecules, simulations.

By the way, what's up with Iterator everywhere?

I want to get the selected number of bins from the slider and pass that number into a python method and do some calculation/manipulation (return: “You have selected 30bins and I came from a Python Function”) inside of it then return some value back to my R Shiny dashboard and view that result in a text field.

Vaex is an open-source DataFrame library which enables the visualisation, exploration, analysis and even machine learning on tabular datasets that are as large as your hard-drive. To do this, Vaex employs concepts such as memory mapping, efficient out-of-core algorithms and lazy evaluations.

Here’s my preferred way of doing it, which uses Keras’s Tokenizer class

vectors here have length 7 instead of 6 because of the extra 0 element at the beginning. This is an inconsequential detail - Keras reserves index 0 and never assigns it to any word.

It’s possible to check whether a variable refers to it with the comparison operators == and !=

>>> x, y = 2, None
>>> x == None
False
>>> y == None
True
>>> x != None
True
>>> y != None
False

>>> x is None
False
>>> y is None
True
>>> x is not None
True
>>> y is not None
False

Python allows defining getter and setter methods similarly as C++ and Java

>>> class C:
...     def get_x(self):
...         return self.__x
...     def set_x(self, value):
...         self.__x = value

>>> c = C()
>>> c.set_x(2)
>>> c.get_x()
2

In almost all cases, you can use the range to get an iterator that yields integers

>>> x = [1, 2, 4, 8, 16]
>>> for i in range(len(x)):
...     print(x[i])
... 
1
2
4
8
16

>>> for item in x:
...     print(item)
... 
1
2
4
8
16

Sometimes you need both the items from a sequence and the corresponding indices

>>> for i in range(len(x)):
...     print(i, x[i])
... 
0 1
1 2
2 4
3 8
4 16

>>> for i, item in enumerate(x):
...     print(i, item)
... 
0 1
1 2
2 4
3 8
4 16

But what if you want to iterate in the reversed order? Of course, the range is an option again

>>> for i in range(len(x)-1, -1, -1):
...     print(x[i])
... 
16
8
4
2
1

>>> for item in x[::-1]:
...     print(item)
... 
16
8
4
2
1

>>> for item in reversed(x):
...     print(item)
... 
16
8
4
2
1

it’s often more elegant to define and use properties, especially in simple cases

>>> class C:
...     @property
...     def x(self):
...         return self.__x
...     @x.setter
...     def x(self, value):
...         self.__x = value

>>> c = C()
>>> c.x = 2
>>> c.x
2

Python has a very flexible system of providing arguments to functions and methods. Optional arguments are a part of this offer. But be careful: you usually don’t want to use mutable optional arguments

>>> def f(value, seq=[]):
...     seq.append(value)
...     return seq

>>> f(value=2)
[2]

>>> f(value=4)
[2, 4]
>>> f(value=8)
[2, 4, 8]
>>> f(value=16)
[2, 4, 8, 16]

Iterating over a dictionary yields its keys

>>> z = {'a': 0, 'b': 1}
>>> for k in z:
... print(k, z[k])
... 
a 0
b 1

>>> for k, v in z.items():
...     print(k, v)
... 
a 0
b 1

following the rules called The Zen of Python or PEP 20

You can use unpacking to assign values to your variables

>>> a, b = 2, 'my-string'
>>> a
2
>>> b
'my-string'

what if you want to iterate over two or more sequences? Of course, you can use the range again

>>> y = 'abcde'
>>> for i in range(len(x)):
...     print(x[i], y[i])
... 
1 a
2 b
4 c
8 d
16 e

>>> for item in zip(x, y):
...     print(item)
... 
(1, 'a')
(2, 'b')
(4, 'c')
(8, 'd')
(16, 'e')

>>> for x_item, y_item in zip(x, y):
...     print(x_item, y_item)
... 
1 a
2 b
4 c
8 d
16 e

None is a special and unique object in Python. It has a similar purpose, like null in C-like languages

Python allows you to chain the comparison operations. So, you don’t have to use and to check if two or more comparisons are True

>>> x = 4
>>> x >= 2 and x <= 8
True

>>> 2 <= x <= 8
True
>>> 2 <= x <= 3
False

Unpacking can be used for the assignment to multiple variables in more complex cases

>>> x = (1, 2, 4, 8, 16)
>>> a = x[0]
>>> b = x[1]
>>> c = x[2]
>>> d = x[3]
>>> e = x[4]
>>> a, b, c, d, e
(1, 2, 4, 8, 16)

>>> a, b, c, d, e = x
>>> a, b, c, d, e
(1, 2, 4, 8, 16)

>>> a, *y, e = x
>>> a, e, y
(1, 16, [2, 4, 8])

the most concise and elegant variables swap

>>> a, b = b, a
>>> a
'my-string'
>>> b
2

Python doesn’t have real private class members. However, there’s a convention that says that you shouldn’t access or modify the members beginning with the underscore (_) outside their instances. They are not guaranteed to preserve the existing behavior.

>>> class C:
...     def __init__(self, *args):
...         self.x, self._y, self.__z = args
... 
>>> c = C(1, 2, 4)

>>> c.x  # OK
1

>>> c._y  # Possible, but a bad practice!
2

>>> c.__z # Error!
Traceback (most recent call last):
File "", line 1, in 
AttributeError: 'C' object has no attribute '__z'
>>> c._C__z # Possible, but even worse!
4
>>>

what if an exception occurs while processing your file? Then my_file.close() is never executed. You can handle this with exception-handling syntax or with context managers

>>> with open('filename.csv', 'w') as my_file:
...     # do something with `my_file`

Python code should be elegant, concise, and readable. It should be beautiful. The ultimate resource on how to write beautiful Python code is Style Guide for Python Code or PEP 8

Python also supports chained assignments. So, if you want to assign the same value to multiple variables, you can do it in a straightforward way

>>> x = 2
>>> y = 2
>>> z = 2

>>> x, y, z = 2, 2, 2

>>> x = y = z = 2
>>> x, y, z
(2, 2, 2)

open a file and process it

>>> my_file = open('filename.csv', 'w')
>>> # do something with `my_file`

>>> my_file = open('filename.csv', 'w')
>>> # do something with `my_file and`
>>> my_file.close()

The author of the class probably begins the names with the underscore(s) to tell you, “don’t use it”

You can keep away from that with some additional logic. One of the ways is this:

>>> def f(value, seq=[]):
...     seq.append(value)
...     return seq

>>> def f(value, seq=None):
...     if seq is None:
...         seq = []
...     seq.append(value)
...     return seq

>>> def f(value, seq=None):
...     if not seq:
...         seq = []
...     seq.append(value)
...     return seq

>>> f(value=2)
[2]
>>> f(value=4)
[4]
>>> f(value=8)
[8]
>>> f(value=16)
[16]

The Pythonic way is to exploit the fact that zero is interpreted as False in a Boolean context, while all other numbers are considered as True

>>> bool(0)
False
>>> bool(-1), bool(1), bool(20), bool(28.4)
(True, True, True, True)

>>> for item in x:
...     if item:
...         print(item)
... 
1
2
3
4

When you have numeric data, and you need to check if the numbers are equal to zero, you can but don’t have to use the comparison operators == and !=

>>> x = (1, 2, 0, 3, 0, 4)
>>> for item in x:
...     if item != 0:
...         print(item)
... 
1
2
3
4

it’s a good practice to use properties when you can and C++-like getters and setters when you have to

The reason that Julia is fast (ten to 30 times faster than Python) is because it is compiled and not interpreted

Scala is ten times faster than Python

First of all, write a script that carries out the task in a sequential fashion. Secondly, transform the script so that it carries out the task using the map function. Lastly, replace map with a neat function from the concurrent.futures module

Python standard library makes it fairly easy to create threads and processes

Python is a poor choice for concurrent programming. A principal reason for this is the ‘Global Interpreter Lock’ or GIL. The GIL ensures that only one thread accesses Python objects at a time, effectively preventing Python from being able to distribute threads onto several CPUs by default

Introducing multiprocessing now is a cinch; I just replace ThreadPoolExecutor with ProcessPoolExecutor in the previous listing

this article merely scratches the surface. If you want to dig deeper into concurrency in Python, there is an excellent talk titled Thinking about Concurrency by Raymond Hettinger on the subject. Make sure to also check out the slides whilst you’re at it

As the name suggests, multiprocessing spawns processes, while multithreading spawns threads. In Python, one process can run several threads. Each process has its proper Python interpreter and its proper GIL. As a result, starting a process is a heftier and more time-consuming undertaking than starting a thread.

its purpose is to dump Python tracebacks explicitly on a fault, after a timeout, or on a user signal

what parts of the software do we profile (measure its performance metrics)

Line profiling, as the name suggests, means to profile your Python code line by line

The profile module gives similar results with similar commands. Typically, you switch to profile if cProfile isn’t available

Another common component to profile is the memory usage. The purpose is to find memory leaks and optimize the memory usage in your Python programs

The purpose of trace module is to “monitor which statements and functions are executed as a program runs to produce coverage and call-graph information

method profiling tool like cProfile (which is available in the Python language), the timing metrics for methods can show you statistics, such as the number of calls (shown as ncalls), total time spent in the function (tottime), time per call (tottime/ncalls and shown as percall), cumulative time spent in a function (cumtime), and cumulative time per call (quotient of cumtime over the number of primitive calls and shown as percall after cumtime)

tracing is a special use case of logging in order to record information about a program’s execution

If a method has an acceptable speed but is so frequently called that it becomes a huge time sink, you would want to know this from your profiler

trace and faulthandler modules cover basic tracing

It’s just that it often makes sense to write code in the order JOIN / WHERE / GROUP BY / HAVING. (I’ll often put a WHERE first to improve performance though, and I think most database engines will also do a WHERE first in practice)

# Creating a 5x5 matrix arr = [[i for i in range(5)] for j in range(5)] arr >>> [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]

arr = [[i for i in range(5)] for j in range(5)]
arr
>>> [[0, 1, 2, 3, 4],
 [0, 1, 2, 3, 4],
 [0, 1, 2, 3, 4],
 [0, 1, 2, 3, 4],
 [0, 1, 2, 3, 4]]

x = [2,45,21,45] y = {i:v for i,v in enumerate(x)} print(y) >>> {0: 2, 1: 45, 2: 21, 3: 45}

x = [2,45,21,45]
y = {i:v for i,v in enumerate(x)}
print(y)
>>> {0: 2, 1: 45, 2: 21, 3: 45}

x = [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]] arr = [i for j in x for i in j] print(arr) >>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

x = [[0, 1, 2, 3, 4],
 [5, 6, 7, 8, 9]]

arr = [i for j in x for i in j]
print(arr)
>>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

arr = [i for i in range(10) if i % 2 == 0] print(arr) >>> [0, 2, 4, 6, 8] arr = ["Even" if i % 2 == 0 else "Odd" for i in range(10)] print(arr) >>> ['Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd']

arr = [i for i in range(10) if i % 2 == 0]
print(arr)
>>> [0, 2, 4, 6, 8]

arr = ["Even" if i % 2 == 0 else "Odd" for i in range(10)]
print(arr)
>>> ['Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd', 'Even', 'Odd']

use pyenv. With it, you will be able to have any version you want at your disposal, very easy.

the __ methods allow us to interact with core concepts of the python language. You can see them also as a mechanism of implementing behaviours, interface methods.

Dunder or magic method, are methods that start and end with double _ like __init__ or __str__. This kind of methods are the mechanism we use to interact directly with python's data model

Dutch programmer Guido van Rossum designed Python in 1991, naming it after the British television comedy Monty Python's Flying Circus because he was reading the show's scripts at the time.

supporting this field is extremely easy If you keep raw data, it's just a matter of adding a getter method to the Article class.

@property
def highlights(self) -> Sequence[Highlight]:
    default = [] # defensive to handle older export formats that had no annotations
    jsons = self.json.get('annotations', default)
    return list(map(Highlight, jsons))

query language doesn't necessarily mean a database. E.g. see pandas which is capable of what SQL is capable of, and even more convenient than SQL for our data exploration purposes.

cachew lets you cache function calls into an sqlite database on your disk in a matter of single decorator (similar to functools.lru_cache). The difference from functools.lru_cache is that cached data is persisted between program runs, so next time you call your function, it will only be a matter of reading from the cache.

We only need to use global keyword in a function if we want to do assignments / change them. global is not needed for printing and accessing.

# This function modifies global variable 's' 
def f(): 
    global s 
    print s 
    s = "Look for Geeksforgeeks Python Section"
    print s  

# Global Scope 
s = "Python is great!" 
f() 
print s