Tensorflow Study Note, Part Three


  1. Interactive Session

tf.InteractiveSession() creates a default session that can be used without explictly called in a IPython environment


a = tf.constant(1)

  1. Regular Session

Regular session needs to be run within a python context or through session object

Example 1:

a = tf.constant(1)
sess = tf.Session()

Example 2:

a = tf.constant(1)
with tf.Session():


This function is used to add bias to the input tensor (element-wise addition between "bias" vector and the feature vector)

It should be noted that the bias added here is completed different from the normal concept of adding a bias to the hidden unit (summed weight before activation)


The run function of tf.Session provides a interface to execute provided TF operations and evaluate Tensors.

Feature Preprocessing

Continuous features can be feed into the first hidden layer of the neural network directly. Discrete features are recommended to go through an embedding layer.


This API accepts a [batch_size, doc_length] tensor of type "int32" or "int64"


This API is used to loopup an embedding using ID.

Categorial to Embedding Pipeline

Categorial Label ---> Ordinal ID ---> One-hot Embedding ----> Dense Embedding

About Immigration

The Cause of Immigration

The life quality of western countries is far better than that of China in terms of economical, environmental and social conditions. I found myself increasingly believe in western cultures and values and that is the primary driving force of my immigration thoughts. I need to improve my language skill as well as professional experiences to get ready for the oppertunity. Meanwhile, I need to take it very carefully to examine this thought to confirm it is a good fit for me and my family.

I need a carefully planning to merge my immigration effort with the long-term paradigm shift of my family earning, that is, to shift from work-based earning to asset-based earning. (Asset here is something that can generate revenue without extra mental/physical labor)

Feature Embedding of Categorial Values

Word Embedding

  1. The embedding of a word is the hidden layer ouput (a vector) got when feeding the one-hot vecotr (input) of the word.
  2. The target value in the training data carries sequential information (word sequence in a sentence)
  3. The hidden layer weight matrix is the word vector lookup table
  4. Word embedding is a by-product of language modelling

Embedding Layer

  • Embedding layer is just a linear layer that transforms one-hot input into embedding matrix (look-up table)

Categorial Embedding

  1. The embedding of categorial data is obtained in model training, just as that of word embedding.
  2. The target value carries prediction information (predicted value), which is opposed to the sequential information in word embedding

Pre-trained Embedding

  • Pre-trained embedding can be used in new model training to improve performance (both accuracy and speed)

Ordinal Values

  1. Ordinal Values that carries information but in discrete (by definition) values should be normalized to 0-1 (or -1 - 1) scale as the input for DNN

Two Sigma Competition


The dataset of Two Sigma Competition is a h5 file. It can be read into a 1,710,756 rows x 111 columns pandas dataframe.

All datapoints are identified by two attributes combined: id + timestamp, both of which are not unique. Id is a financial security and timestamp indicates the time of quote (Y).

In the dataset there are 1424 different ids and 1813 different timestamps. In general, the data point is sampled in a fixed timestamp interval of 750.

Pandas Notes

DataFrame Query Method


  1. df.query("name==ok")
  2. df[df.id==12].count()

// Return a slice copy of the DataFrame based on condition
// indexing operators [] and attribute operator .


  1. series.where(id==3)

Returning a VIew v.s. Copy

  1. Return a copy : df[0]["ccc"] // this is chain indexing
  2. Return a view: df.loc[0, "ccc"]

DataFrame Apply Function

  1. df.apply() can apply specified operation on the dataframe object

Added a new column to the DataFrame based existing columns

  1. df = df.apply()
  2. df = df["one_column"].apply()

Pandas Groupby

  1. df.groupby('Team').groups
  2. tc.groupby(['Pclass'])['Survived'].mean()

English Vocabularies

  1. progdigious
    very impressive
  2. Earnest
    very serious mentally
  3. Aloof
    keep distance from things
  4. Rupture
    break relationship with
  5. Demeanor
    the appearance to someone, the attitude to someone
  6. Shephered
    To guide
  7. Ostensible
    intended to display but not true
  8. Articulate
    express oneself clearly
  9. loath
    unwilling to do something againest one's own thinking
    to give comfort in grief or misfortune
  10. Tranquil
    quiet and peaceful
  11. secular
    of or relating to physical world, not spiritual
  12. infidel
    a person who does not believe in religion
  13. Marvelous
    extremely good or enjoyable
  14. Rudimentary
    very basic
  15. Renaissance
    A period of time in Europe when culture and science are in a boom
  16. Scared
    worth of respect in a religion
  17. Embroil
    to throw someone into disorder
  18. Renown
    highly respected
  19. Unanimous
    Agreed by a crowd of people
  20. Injunction
    an order from court or other authority that something must be done
  21. Dissent
    To publicly disagree with opinions
  22. Proscribe
    Prohibit from doing something
  23. Deviate
    to stray from a principle or norm
  24. Cosmopolitan
    a person who have been different part of the world and know a wide range of things
  25. Disseminate
    to cause to go to many people / spread knowledge
  26. Vigilant
    Carefully watchful to avoid danger
  27. Inculpatory
    Imply guilt
  28. Exculpatory
    to clear from alleged fault or guilt
  29. Consensus
    widely consent inside a community
  30. Scum
    A layer of something unpleasant or unwanted that forms on top of a liquid
  31. Revelation
    a usually secret fact that is made known
  32. Barrage
    a bar placed at a water course to increase the depth of the water
  33. Collision
    secret cooperation with someone for a dishonst purpose
  34. Subpoena
    a written order that command someone to appear on court
  35. Obsessive
    thinking about something or someone in a way that is too much
  36. Solidarity
    A feeling of unity between people who have the same interests, goals , etc
  37. Secular
    Not overtly or specificially religious

how to deal with situations when team members are not qualified enough to do the job on schedule, is a problem faced by many mangers in technology field. It is not without budget to hire qualified developer, but not enough prospective canadaicates to interview with. The technology industry is experiencing dramatic shortage in skiled manpower.