A novel flow-based attention mechanism, attention flow, was presented to discover latent causal connections under observations and …

Online Video Streaming services such as Hulu hosts tens of millions of premium videos, which requires an effective recommendation …

Recently, the embedding-based recommendation models (e.g., matrix factorization and deep models) are prevalent in both academia and …

A generalized backpropagation framework, Backprop-Q, was introduced to address the issue of gradient propagation on stochastic …

I would rather see intelligence as an ability to grow in knowledge and wisdom than as a product made by genius engineers that is ready for sale. If we really wish AI to own the same intellectual ability as us, we should be aware of the process of how we develop our intelligence. We, as intelligent beings, are born as a blank sheet of paper. To acquire intellectual growing, children need to learn from the environment, parents, teachers, friends and even opponents except engineers, that is, children need to learn from interaction with other humans. From this point of view, changing children with AI, I am curious about what roles humans could play in the long and continuous process of growing AI except acting as engineers and scientists to built AI …

AI needs deliberately designed education. Current machine learning and deep learning models are trained by tasks that come mostly from real-life applications with practical purposes. Few tasks are designed specially and systematically for the purpose of studying pure intelligence. Why is this very important? Much efforts today in the AI community have been put on designing AI models that are AI’s brains, whereas less studies are concentrated on how to design and organize a series of tasks that can help AI grow …

How are we motivated to do continuous self-training? Each time I attempt to see intelligence in machine learning models, there would be a doubt rising in my head - how could intelligence, characterized by high-level abstraction capability, be acquired simply by a gradient descent process reaching convergence? I would rather believe that the journey of pursuing intelligence is a back-and-forth, twist-and-turns, and chaotic-and-conscious process with loops, eventually jumping out of loops and reaching some enlightment point. Therefore, it cannot be achieved only with one single objective formulated by one specific optimizatoin problem. Then, I acquired some inspiration from children learning to propose a hypothesis of a continuous self-training framework - a continuous process alternating between finding a puzzle and solving a puzzle …

Two theories collaboratively characterize the brain functioning - modularity and distributive processing. One of the keys to the success of artificial neural networks is the idea of distributive representation. However, the modularity and specialization property has not been well taken into account. I tend to think of a multi-way graph-structured architecture, with branching or mergining possibly at any point, an ultimate exention to skip connections, lateral connections and highway. Moreover, this gaint net would not be operated fully, but only a small fraction would be actived to form a computation subgraph. This implies a navigation problem that requires choosing nodes and edges at each step, a type of actions that happens in the computation graph. Finally, the computation graph may form a high-level conceptual or imagination world. I speculate that it might be attention that causes our consciouness flow. On the one hand, there must be an attention mechanism to draw out a subgraph and navigate the computation flow; on the other hand, attention can be seen as a mental action, consciously navigating us in our imagination world …