Syncing bash history across sessions

export PROMPT_COMMAND="history -a; history -n"

I always work with multiple sessions and have from 5-10 sessions. I work on one session and then switch to another session, I try to do Ctrl + R , however since the bash history are not synced, I need to go to that session and then copy the command.
The more frustrating encounter is when you remember the keywords of the statement but used it long back. You will surely lose the history, being lots of sessions un-synced.

The above command provides just that – sync bash history from multiple sessions of bash.

  • PROMPT_COMMAND when a new command is executed, commands in this variable is executed. Commands in this variable is executed every time bash prompt is shown.
  • history -a appends new command statement to bash history file
  • history -n statement to read any unread lines from bash history file to current session

Rust : Important Concepts

Void Functions vs Diverging Functions

There are two kinds of functions that doesn’t return any value

#1.  fn void_function() {} 

#2.  fn diverging_function() -> ! {}

#1 A Void function doesn’t have any any return doens’t mean it doesn’t return any value, it returns () type. This is equivalent to void in C/C++.

fn void_function() -> () {}

#2 A diverging function is guarenteed to not return anything ie. println!("sample text");

Example of diverging functions is 

fn diverging_function() -> ! {
   panic!()
}

fn diverging_function_2() -> ! {
    loop{}
}

Comparing with other languages, its similar to returning Exception in other languages. In languages like Java, even if the function return some data type, and due to exception it returns some Exception. The data type is different from what is output, since java doens’t have a way to mark it as actual method as diverging. Instead, Rust has special construct to specify nothing is returning ie. specify the function as diverging, which has similar effect.

public String getString(String val){
     if (val == null){
           throw Exception();
     }
     return new String(val);
}

Next: Stack Unwinding

Saving trained model

In most cases, for training the model with the dataset we have is very time consuming and also processing hungry job which is costly task. To test in our development environment we have to do is test the trained model or use the trained model for in production without going for multiple training.

If you have done some ML project you would have understood, how time and processor consuming task it is even when done in GPUs. For a application to use the model and train each time the application runs is unacceptable, so we can save the current trained state of the model for later use without of retraining the model on the same dataset again and again.

We can accomplish this in python using some packages like

  • Pickle (Python Object Serialization Library)
  • Joblib (One of the Scikit-learn Method)

Pickle?

You might have heard this term somewhere when you go though ML articles or doing projects. This library is popular for Serialization(Pickling) and Marshalling (Unpickling). Pickling is the process of converting any Python object into a stream of bytes in hierarchy.Unpickling is process of converting the pickled stream of bytes to original python object following the object hierarchy.

Example:

Serialization (Pickling)
import pickle

pickle_file = 'string_list_pickle.pkl'
names = ['apple', 'ball', 'cat']

store_pickle = open(pickle_file, 'wb')
pickle.dump(names, store_pickle)
store_pickle.close()

Continue reading “Saving trained model” →

Simple Neural Network for absolute beginners

This is my first post regarding artificial intelligence (AI). But I promise to include as much as I can from understanding Simple Neural Network (NN) to deep learning through little theoretical but lots of practical implementations. I will also include simple projects where possible. Lets begin building then.

I like Python a lot so most of the works will be done in Python. Later on I am hoping to develop them in Java and Scala.
To learn NN we will not be using any NN libraries but some mathematical libraries, ie. numpy.

Learn basics of Numpy HERE .

To begin building NN which is supposed to mimic how our brain works, we have to understand little bit of our own Brain.
An averaged sized Brain includes of 100 billion neurons connected by synapses. Neurons are the basic unit of brain which plays major role for all the tasks done by brain.  Blah blah blah… its better you go through this well written article (A Basic Introduction To Neural Networks).

This tutorial we will be building a Artificial unit of this very Neuron. I consider you know matrices which we will be using as mathematical foundation for building NN with numpy.

Our simple ANN will include three inputs and a output. (Input: 3, Output: 1). This neuron we will build should classify a basic problem of classification. We will use various different training algorithms to train our neuron for classification.

So our neuron will have very small dataset for training (a deeplearning model will need very very large dataset for better performance) which will be enough in this problem.

Example# Input A Input B Input C Output Y
1 0 0 1 0
2 1 1 1 1
3 1 0 1 1
4 0 1 1 0
5 1 0 0 ??? (1 expected)

So what will be the output for the last data row (Row #5)?.

Continue reading “Simple Neural Network for absolute beginners” →

Bloom filter – Python

A Bloom Filter is a space-efficient probablistic data structure, to test whether an element is a member of a set. This algorithm for data representation ensures the element is in the dataset or not. This may have false positive matches but not false negative result.

The most common use of Bloom Filter Algorithm is to see if an elements is in the disk before performing any operations. This reduces the I/O for lookups dramatically over large datasets.

Consider we have to check a email address to search from large dataset of emails of millions. Searching all the emails in memory definitely is very inefficient and takes lots of time. We can create bloom filter bitarray, which is very small compared to the original dataset and has almost same required result. This is what used in Yahoo Mail. When you log into Yahoo mail, the browser page requests a bloom filter representing your contact list from Yahoo servers. The Bloom Filter is compact and easily fits in your browser cache. In Yahoo, email is verified if it is in existed in your contact list.

Another scenario, consider we have to get unique counts in the dataset, then we can use bloom filter to test if certain pattern or element has already been seen or not in the dataset. Of course this creates some false positives, but this might be much efficient than to compare everything from the memory.

Apache HBase uses bloom filter to boost read speed by filtering out unnecessary disk reads of HFile blocks which do not contain a particular row or column.

Quora implemented a sharded bloom filter in the feed backend to filter out stories that people have seen before. It is much faster and more memory efficient than previous solutions (Redis, Tokyo Cabinet and DB) and saves hundreds of ms on certain types of requests.(Tao Xu, Engineer at Quora)

Transactional Memory (TM) has recently applied Bloom filters to detect memory access conflicts among threads.(Mark Jeffrey, modeled Bloom filters for concurrency conflict detection).

Not to mention Facebook ( Typeahead Search Tech Talk (6/15/2010) ),, LinkedIn ( Cleo: the open source technology behind LinkedIn’s typeahead search | LinkedIn Engineering),  Bit.ly (dablooms – an open source, scalable, counting bloom filter library ) have implemented their own Bloom Filter.

More examples ? Go here https://en.wikipedia.org/wiki/Bloom_filter#Examples

Ok then, enough of the usage of Bloom Filter. We will be developing our own for searching email address from a list in Python.

Continue reading “Bloom filter – Python” →

Singleton Design Pattern

Singleton pattern restricts the instantiation of a class and ensures that only one instance of the class exists.

public class Singleton {
	private static Singleton instance;
	private Singleton() {}
	public static Singleton getInstance() {
		if (instance == null) {
			instance = new Singleton();
		}
		return instance;
	}
}

The above code is the general class definition of Singleton Design Pattern.

The above code guarantees the single object creation of the class. The only way to retrieve an instance of the class is to call getInstance() method.

 

Writing Chatbot?

Ok this is a very interesting and hot topic right now. I personally have been so in love with this trend. I have developed a simple  Bot in my college using AIML technology. The similiar is being used in most of the bot right now also, but in different context. After Facebook has created a new development platform for bot developers. Its even being more interesting in case of usage.

“Bot for everything and everyone.” This is how I see.

Enough of the literature. 😀

I have been collecting materials and developing bots for basic tasks for my personal usage. And before creating bot I would suggest understanding branches required for bot development, ie. Flow of NLU.

The main task that a bot building starts with is Intent Classification and Entity Extraction from the message text given to the bot by user.

Example:
User -> Bot:
 Book me a room for a night. 

 Intent/ Context: book_room
 Entity Extracted: number_of_room -> a room (one room)
duration -> a night (one night) Continue reading “Writing Chatbot?” →

Database programming (SQLite)

Beginning with SQLAlchemy Python

Ok I have been doing Python for a while now and Python is amazing. So thought of writing a blog about it in my new blog. Leaving the chitchat.

We will be performing database operations (CRUD ) in SQLite database. This is a simple database that many programming language support and recommended for beginning database programming for most of them. We will be building a simple todo application to begin with. For further practice, you can create Restaurant Menu App, Remainder App, whatever you like. Be creative. 😎😎😎

We have few steps that we will be following. Importing libraries, connecting to database (*.db) and creating session interface for performing operations.

We will be using SQLAlchemy python module to begin.

Importing dependencies

import sys
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
from sqlalchemy import create_engine

These are the dependencies that we will be using for begin with.

Continue reading “Database programming (SQLite)” →