UnicodeEncodeError exception in python

Error message:
UnicodeEncodeError: codec can't encode character  in position : ordinal not in range(128)

Python represents the string in utf-8 though the string has charachters from different encoding, hence the error message.

Scenario: Have charachters in string which are represented as string(utf-8) but has to be unicode

• Check the type, that should tell what's the encoding:

           type(str_var)

• If the type is 'str': you should be decoding from utf-8 to unicode:

           str_var.decode('utf-8')

• If the type is unicode, then encode it to utf-8

           univar.encode('utf-8')

Note: if the encoding is wrong you might end up with no value if you do to avoid the exception:
          str_val.encode('ascii', 'ignore')

Refer unicode link to understand more about unicode.

Unix/Linux Shell commands

Unix/Linux Shell commands

Shortcuts

!!  (bang bang): Run entire previous command line.

    Previous command:
        $ ls -l /usr/bin/g++ /usr/bin/gcc
          -rwxr-xr-x 4 root root 221736 Oct  7  2010 /usr/bin/g++
          -rwxr-xr-x 2 root root 221920 Oct  7  2010 /usr/bin/gcc
     Shortcut:
        $ !!
           ls -l /usr/bin/g++ /usr/bin/gcc
           -rwxr-xr-x 4 root root 221736 Oct  7  2010 /usr/bin/g++
           -rwxr-xr-x 2 root root 221920 Oct  7  2010 /usr/bin/gcc

!^  (bang caret): Run just the command/program from previous command line

     Previous command:
         $ echo "contents of file1" > file1 ; echo "contents of file2" > file2

         $ ls file1 file2

            file1  file2

      Shortcut:

         $ cat !^
            cat file1
            contents of file1

!* (bang star): All arguments of previous command line

    Previous command:
         $ echo "contents of file1" > file1 ; echo "contents of file2" > file2

         $ ls file1 file2

            file1  file2
    Shortcut:
         $ head !*
            head file1 file2
            ==> file1 <==
            contents of file1

            ==> file2 <==
            contents of file2

Navigating/edit command line on terminal

Ctrl + A or Home	Moves the cursor to the start of a line.
Ctrl+ E or End	Moves the cursor to the end of a line.
Esc + B	Moves to the beginning of the previous or current word.
Ctrl + K	Deletes from the current cursor position to the end of the line.
Ctrl + U	Deletes from the start of the line to the current cursor position.
Ctrl + W	Deletes the word before the cursor.
Alt + B	Goes back one word at a time.
Alt + F	Moves forward one word at a time.
Alt + C	Capitalizes letter where cursor is and moves to end of word.

Commands

ls : list directory contents

          1. List the files by directories but not recurse with -l (minus l) flag
              ls -ld *

          2. Find timestamp of a file in seconds:
              ls -l --time-style=full-iso

Copy files across sites/machines

scp <machine name>:<source file path>  <machine name>:<Destination path>
Note: Machine name is not necessary if you are already logged into that machine

reset: Reset the terminal

• Clears the garbled terminal with illegible characters 

locale: Displays the current locale settings

More Unix tricks here
http://lifehacker.com/5743814/become-a-command-line-ninja-with-these-time-saving-shortcuts
https://help.ubuntu.com/community/UsingTheTerminal

Settings

setenv LC_ALL C

If there are few illegible characters displayed in place of special charachters in files or man pages set the above to make them visible properly.

Learn locale here, here

Install XV Image viewer on Ubuntu linux

'sudo apt-get install xv' doesn't install xv on Ubuntu, you would need to build it.

Follow the steps as in here to build, you might need to do few more changes for tiff make errors which I encountered.

sudo bash

cd /tmp

wget ftp://ftp.trilon.com/pub/xv/xv-3.10a.tar.gz

wget http://www.ulich.org/hints/resources/xv-3.10a-jumbo20050501-1.diff.gz

wget http://www.ulich.org/hints/resources/xv-3.10a-jumbo-patches-20050501.tar.gz

tar xvzf xv-3.10a.tar.gz

tar xvzf xv-3.10a-jumbo-patches-20050501.tar.gz

gzip -d xv-3.10a-jumbo20050501-1.diff.gz

cd xv-3.10a

patch -p1 < ../xv-3.10a-jumbo-fix-patch-20050410.txt

patch -p1 < ../xv-3.10a-jumbo-enh-patch-20050501.txt

patch -p1 < ../xv-3.10a-jumbo20050501-1.diff

for i in libxt-dev libc6-dev xlibs-dev libjpeg62-dev libtiff4-dev libpng12-dev libxt-dev; do apt-get install $i; done

make

cp xv /usr/local/bin/

For Make errors like:

xvtiff.c:15:79: fatal error: tiffio.h: No such file or directory

compilation terminated.

and

/usr/bin/ld: cannot find -ltiff

collect2: ld returned 1 exit status

Add the xv extracted path to TIFFINC and TIFFLIB in Makefile: ( I had extracted to /tmp/xv-3.10a )

TIFFINC = -I$(TIFFDIR)/include -I/tmp/xv-3.10a/tiff

TIFFLIB = -L$(TIFFDIR)/lib -L/tmp/xv-3.10a/tiff -ltiff

PS:

Official Site of XV is http://www.trilon.com/xv/