Python Programming Notes

Language Features

  1. Context
    The variables defined within a context can be accessed outside of the cotext.
    The context is used to apply automatic operations.

Escape Depth

  1. Python Interpreter will escape special characters only once !
    \n ---> r"\n"

  2. Unicode Escape Characters
    Python will treat \x0343 as unicode string when printing to console or write to text file.

Python will treat \x0343 as ASCII characters when printing to console or write to text file (!!! Due to The Escape Only Once Principle)

  1. Unnecessary Escapes can be removed by line = line.replace(r"\", "\")

Python Json String "()"

(1,2,3) ---> String Repr: "(1, 2, 3)"

Nohup下输出重定向失效的问题

Comment: It looks like you need to flush stdout periodically (e.g. sys.stdout.flush()). In my testing Python doesn't automatically do this even with print until the program exits.
You can run Python with the -u flag to avoid output buffering
1. 解决办法之一:加-u
2. 解决办法之二: sys.stdout.flush()

Pandas Notes

DataFrame Query Method

DataFrame

  1. df.query("name==ok")
  2. df[df.id==12].count()

// Return a slice copy of the DataFrame based on condition
// indexing operators [] and attribute operator .

Series

  1. series.where(id==3)

Returning a VIew v.s. Copy

  1. Return a copy : df[0]["ccc"] // this is chain indexing
  2. Return a view: df.loc[0, "ccc"]

DataFrame Apply Function

  1. df.apply() can apply specified operation on the dataframe object

Added a new column to the DataFrame based existing columns

  1. df = df.apply()
  2. df = df["one_column"].apply()

Pandas Groupby

  1. df.groupby('Team').groups
  2. tc.groupby(['Pclass'])['Survived'].mean()

Django Apache MODWSGI配置问题

问题

httpd: Syntax error on line 491 of /home/ubuntu/httpd/conf/httpd.conf: Cannot load /home/ubuntu/httpd/modules/mod_wsgi.so into server: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory

解决方法

sudo ldconfig /home/ubuntu/miniconda3/lib

Python Programming Engineering Guidelines

Principles:

Install executable package into a directory (example: /apps/my-app) instead of site-packages

Style

Script Python
1. Keep all statements, functions, classes in one module (your script !) . No relative imports is allowed.
2. function main() is not needed.

Package Python (package name maintained)
1. Keep the overall package layout flat. The package name is the only entry point of all module imports, use absolute imports only, sub-packages are generally accepted but not encouraged.
example:
from mypackage.mymodule import myfunction
2. Expand sys.path in the module that contains main()

Package Python (package name not maintained)
1. Keep the overall package layout flat. Use relative imports (import from the same folder) only, sub-packages are generally accepted but not encouraged.
example:
from mymodule import xxx

Recommendations

  1. Coding scripts (temporary):
    Put all definitions, variables, classes into one script; Or split them into several scripts if really needed (using relative import)

  2. Coding non-trivial packages:
    Split definitions, variables, classes into separate scripts; using absolute import, expand sys.path if needed.

Problems:

  • Interfaces cannot have typed parameters (dynamic typing nature), which raises troubles and confusions for the users (partial due to the lack of IDE support).
  • Interfaces with a generics nature in dynamic typing system accepts arguments with different primitive types (root types). This complicates interface calls.