Kafka installation on ubuntu

Installation for Ubuntu
Instruction link: http://kafka.apache.org/documentation.html
Commands
Install Java
https://help.ubuntu.com/community/Java

https://www.digitalocean.com/community/tutorials/how-to-install-java-with-apt-get-on-ubuntu-16-04

sudo apt-get update

sudo apt-get install default-jre

sudo apt-get install default-jdk

wget http://redrockdigimark.com/apachemirror/kafka/0.10.0.0/kafka_2.11-0.10.0.0.tgz

tar -xzf kafka_2.11-0.10.0.0.tgz

cd kafka_2.11-0.10.0.0

Sqlite Commands

sqlite3 /path/to/file/wall_test_test.db

Desc Table:
PRAGMA table_info([store_workspace]);
select * from store_workspace;

Delete rows from table:
delete from store_workspace;

List all tables:
.tables;

Ubuntu commands

1. When following messages are seen when logged into Ubuntu:

88 packages can be updated.
80 updates are security updates

Run the following commands to update the packages:

sudo apt-get update

sudo apt-get upgrade

sudo apt-get dist-upgrade

sudo reboot - If necessary

AWS - puTTY Setup, tunneling to VNC Viewer - Install Django

Connection through putty

1. Download puTTy and puTTygen from puTTy download page
2. Convert .pem to .ppk which puTTy understand
1. Start puTTygen
2. Select SSH-2 RSA under "Type of key to generate"
3. Click load, locate the .pem file [ Select the option to display 'All files (*.*)' ]
4. Click "Save private key"
5. Specify the same name for the key that was used for the key pair. PuTTy automatically adds the .ppk file extension
3. Start puTTy
4. In Category pane,
1. select session
2. In 'Host Name' enter user_name@public_dns_name for ubuntu it is 'ubuntu'
3. Select SSH under Connection type
4. Port is 22
5. In the Category pane, expand Connection, expand SSH, and then select Auth
1. Click Browse
2. Select .ppk file and then click Open
3. If you plan to start this session again later, you can save the session information for future use. Select Session in the Category tree, enter a name for the session in Saved Sessions, and then click Save.
4. Click Open to start the PuTTY session

Connect VNC

1. Follow the steps at link
2. Load the above session by clicking Category->Session->select the session under 'Saved session' -> Load
3. In Category pane, expand SSH, select X11, check Enable X11 Forwarding
4. Select Tunnels from Category pane, Enter these values:
1. Source port: 5900
2. hostname:port_number
To determine the destination port, use the number (#) you were given in the vncserver command above and add it to 5900. For example, if your server is running on :1, use hostname:5901
3. Click Add
5. Save the session: Category->Session->select the session under 'Saved session' -> Save
6. Open VNC Viewer
1. VNC server: localhost
2. Click OK

Debug Build Failures on Linux platform

Tools/Unix commands which helps debug build failures

1. nm
2. file
3. objdump
4. strings
5. ar
6. ldd
7. Know about the significance and behavior of different types of symbols.

vim commands/shortcuts

Vim Basics

• Save and Exit (Command mode)

          1. 'w'  : Save the file with latest changes but not exit out of editor

          2. 'wq': Save the file and exit.

          3. 'e'   : Refresh the file if the file has undergone modification from a different process.

                      Note: Any changes to that file from the editor will be lost.

• Navigation keys

          1. 'Ctrl f' : One page forward

          2. 'Ctrl b' : One page backwards

          3. 'w' : Move forward by 1 word

              'W' : Move forward until next space character

          4. 'b' : Move back 1 word

              'B' : Move back to previous space character.

• Search

          1. 'Shift 8' /  '*' : Search the word under cursor

• Increment and Decrement number ( normal/edit mode ):  

          'Ctrl-a' and 'Ctrl-x'

• Add/modify a column of file (vertically)

          Follow the steps:

          1. 'Ctrl-v' : Creates visual block.

          2. Navigate up/down the lines for which you need add/modify. ( Still in visual mode)

          3. 'Shift i'

          4. 'esc'

             

• Windows (commandline mode):

          'vsp <filename>' : Vertical Split
          'sp  <filename>'   : Split window horizontally

          Navigation keys:

           'Ctrl-ww' : Rotate between windows
           'Ctrl-w<arrow keys>' : Navigate to corresponding window as per the arrow key. [ eg: right arrow to move to right window from current line in the current window]

• Tabs (commandline mode):

           'tabe <filename>': Will open the file in a new tab.

          Navigation keys:

           'gt' : Move to next tab. Rotates to first if it's the last tab.
           'gt <number> : Moves to the tab with the given number. [ Ex: 'gt 2' will goto the second tab]

Optimizing file operations efforts using 'vim'. 

Modify multiple files with "similar" operation

This is achieved by recording the operation and recursing that over all the files.

1. Open all the files in vim together. 'vim file1 file2 file3.. file4'  ( If you have a file with list of all the filenames, you can open as: vim `cat file_with_paths` ( notice that back quotes are used here!)
2. Start the first recording: Type 'q' followed by <any character> in command mode.That particular character will be the name of what is being recorded.  Example: 'qs'. All the vim operations you do or commands you run, will start getting recorded now.
3. Do the "similar" operation. Lets consider substituting all the occurrences of  the word "BAD" with "GOOD" as "similar" operation. In command mode, type '%s/BAD/GOOD/g'
4. Save the file and move to the next file: 'wn'  ( 'w' for save, 'n' is to open next file.)
5. Now, run the recording. '@' followed by <any character> will run the recording. In our example, its '@s'. Basically, this particular recording is calling itself! which makes it recursive.
6. End the recording: Type just the single character 'q' to end recording.We have the first recording encapsulated inside itself along with the commands which need to be run for all the files.
7. Run the recording to finish: Type '@s'. Now, all the files having the word 'BAD' will have replaced with the word 'GOOD', in a jiffy :)

Vim Sessions

Create session

:mksession ~/mysession.vim

source that vim file to get the old session

:source ~/mysession.vim

$ vim -S ~/mysession.vim

Amazon aws - view ubuntu desktop using VNC

Setting up VNC on Ubuntu in Amazon EC2
Install the desktop packages and remote desktop (VNC) server:

sudo apt-get update
sudo apt-get install ubuntu-desktop
sudo apt-get install vnc4server

Configure the VNC server enter vncserver, which will then prompt you to create a password. Once created, entervncserver -kill :1 to stop the server.

edit the config file: ~/.vnc/xstartup.

Uncomment the two lines below the string "Uncomment the following two lines for normal desktop." And on the second line add "sh" so the line reads exec sh /etc/X11/xinit/xinitrc. 

Start the VNC server again by entering vncserver. 

Open VNC through Tunneling putty :

References:
http://crl.ucsd.edu/handbook/vnc/