GPU Notes

Theano

Set GPU Device

device=gpu
device=gpu01
device=gpu02
..

GPU Multi-task Running Model

Everytime when new gpu computation program instructions is submitted into the GPU device, it will suspend the on-going task and execute the new instructions (program) immediately

CUDA Benchmark

Purpose

The purpose of this benchmark is to prove that parallel computing on gpu does significantly improves program performance in terms of speed. Also, this benchmark gives an estimate of the performance increase.

Experiment 1

Environment

Alienware 14R2 (i5-6GB-GT650M)
Ubuntu14.04 CUDA7.5 GCC4.8.2

Code

C++11
CUDA

Time Elapsed:

C++11: 5418.464355 millsec
CUDA: 229.8250 millsec

Performance Evaluation

C++11: CUDA = 23.6
CUDA Time Percentage = 4%

Experiment 2

Environment

Alienware 14R2 (i5-6GB-GT650M)
Ubuntu14.04 CUDA7.5 GCC4.8.2
Tensorflow 0.9
Tensorflow 0.8

Code

Python2.7 with Tensorflow 0.9 CPU
Python3.5 with Tensorflow 0.8 GPU

Time Elapsed:

CPU: Time Elapsed: 298.434136 s
GPU: 144.865046 s

Performance Evaluation

CPU: CUDA = 2.06
GPU Time Percntage = 48.5%

Linux Bash Shell(Script)

Basic Concepts

  1. Pipe
    Represented by symbol: "|", pipe directs the stdout of the last command to the stdin of the next command.
    Example (directs the result of find command to grep command):
    find ./src/* | grep .java

  2. Environmental Variable
    export ABC=$(find ./src/* | grep .java)

Arguments in Bash Commands

Arguments in bash commands are all considered as string value. However, double quotations is optional and often omitted. Essentially, bash variables are character strings but arithmetic operations is allowed through "let" tool.

Stat

stat -x abc.txt

Software Engineering Study Notes

Productivity v.s. Complexity

Over engineering happens when the productivity gained does not worth the effort of work and the complexity added into the system.

Memory Safety

A concern in software development that aims to avoid software bugs that cause security vulnerability dealing with random-memory-access, such as buffer overflows and dangling pointers.

Type Safety

Type safety is the extent to which a programming language discourages or prevents type errors.

Java Study Note

Static Variable is Evil

A static variable is essentially a global variable (in Java sense). All methods in the program can potentially change the state of a static variable, which makes it hard to reason and control. The predictability of the overall program is then impaired.

Optional Type

Optional type is created to avoid null return value of method call. Since in Java null is not a sub-type of any other type, return a null would raise a nullpointer exception which crashes the problem.

Time Complexity of Java Collection Methods

| Arraylist | LinkedList

get(index) | O(1) | O(n)
add(E) | O(n) | O(1)
add(E, index) | O(n) | O(n)
remove(index) | O(n) | O(n)
Iterator.remove() | O(n) | O(1)
Iterator.add(E) | O(n) | O(1)

Threading Overhead

Time cost for creating a new thread: 1 ms - 8 ms

Parallel Effectiveness Index:

Tc = Thread Creation Cost
Tr = Single Threading Runtime
N = Number of Threads

equilibrium:
Tr = (nn/(n-1))Tc

Threading is beneficial for a program starting from 18 ms (running time) for Intel i7 Process (8 hardware thread)

Static Member and Static Context

  1. Non-static field can not be referenced from a static context
  2. Non-static methods can not be referenced from a static context
  3. Static field can be referenced from a non-static context
  4. Static methods can be referenced from a non-static context
  5. Static member is not recommended to be accessed via instance reference

Java Classpath

Java classpath are search path of java classes/jar/properties. Only .jar file path and directory path can be taken as the classpath (as the same in C/C++ header search path). Multiple classpaths are separated by colon (:). For classpath of jar files, a wildcard (*) is provided to represent all the jar files in a directory.

Java Null Pointer

A java null pointer belongs to the null type, which has no name. Null in java can be considered as a keyword/literal to represent null reference. Null is not a subtype of anything, nor does it is the supertype of anything.

Microsoft Malmo Note

Agent Command

  1. Method host.sendCommand() signals the beginning of a continuous action. In most cases, the stop of a continuous action requires the terminate signal (such as move 0).

Absolute v.s Continuous Commands

MissionSpec.allowAllAbsoluteMovementCommands()

ContinuousMovementCommand
DiscreteMovementCommand


NLP Terminologies

  1. Lexical Information
    Information relating to the word itself (big, small, large, etc) in addition to the word classes (part-of-speech tags).

  2. Endocentric and Exocentric
    A grammatical construction (e.g. a phrase or compound word) is said to be endocentric if it fulfills the same linguistic function as one of its parts, and exocentric if it does not.

  3. Isomorphic
    corresponding or similar in form and relations.

  4. Context-Free Grammar
    The grammar described in a phrase structure.

Math Notation Conventions

Superscript and Subscript

Superscript refers to the index of training examples
Subscript refers to the index of vector elements.

Sequence Notation

A sequence may be named or referred to by an upper-case letter such as "A" or "S". The terms of a sequence are usually named something like "ai" or "an", with the subscripted letter "i" or "n" being the "index" or the counter. So the second term of a sequnce might be named "a2" (pronounced "ay-sub-two"), and "a12" would designate the twelfth term.

The sequence can also be written in terms of its terms. For instance, the sequence of terms ai, with the index running from i = 1 to i = n, can be written as:
http://www.purplemath.com/modules/series.htm

Hat Operator

In statistics, the hat is used to denote an estimator or an estimated value, as opposed to its theoretical counterpart. For example, in the context of errors and residuals, the "hat" over the letter ε indicates an observable estimate (the residuals) of an unobservable quantity called ε (the statistical errors).

Prime Notation

Derivative Operator

Linux Tool Note

  1. TCP/UDP Connection Test
    nc -zv 127.0.0.1 25331

  2. Show all process
    ps aux | less

  3. Uncompress gzip file
    tar -zxvf {file.tar.gz}

  4. List all hardware information
    lshd

  5. Grep Multiple Patterns
    grep -E '123|abc' filename

  6. Show CPU Information
    cat /proc/cpuinfo

  7. Check if a Process Exist
    ps -ef | grep deplearning

  8. Server Benchmark
    ab -c 1000 -n 50000 http://localhost:8080/

  9. 查看系统日志

tail -f /var/log/syslog

  1. 查看文件夹大小
    du -hs /path/to/directory

Word Embedding

Two ways of modeling sentences

  1. s = [x, y, z];
    x, y, z represents three slots in the sentence.
    Three dimensions

  2. s = [x0, x1, x2, ... xn]
    xn represents any words in the vocabulary, value represents its existence in the sentence.