Posts

Chrome - Chrome appear out of screen / offscreen

I hate Chrome when moved out of screen and you cannot move it back. To fix that,

1. Open the Chrome window (it will be off screen somewhere)
2. Press ALT + Spacebar + X

Then above will maximize your Chrome back to your primary monitor.

Hive - URISyntaxException: Relative path in absolute URI

Problem
I encountered the following exception when I tried to start up Hive

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

Solution
This is configuration in hive-site.xml. Open up hive-site.xml and look for ${system:java.io.tmpdir}/${system:user.name}. If found, replace them with a proper value, e.g. /tmp/somedir. After that, run Hive again.

Hive - Hive metastore database is not initialized

Problem
I encountered the following error when I tried to start up Hive

Exception in thread "main" java.lang.RuntimeException: Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql)
Solution
a. Delete the existing $DERBY_HOME/data/metastore_db with rm command

b. Stop the Derby server

c. Run schematool -initSchema -dbType derby

Hive - Permission denied when trying to start Hive with other users

Problem
I had installed Hive and the Hiver CLI work with the user who installed Hive. However, when I try to start Hive CLI with another user, it failed with

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: Permission denied
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: java.io.IOException: Permission…

Hadoop - Installing Hive with Derby

Image
Summary
This is my personal experience and summary steps to install Hive and Derby

Pre-requisite
Download HIVE from Apache (I am using Hive 2.0.0)Download Derby from Apache (I am using Derby 10.12.1.1)Make sure Java 1.7 is installedHadoop is configured and working (I am using Hadoop 2.7.1)
Installing Hive
1. Move the Hive installer to a directory. For this example, I had create a folder /usr/local/hive for Hive

cp apache-hive-2.0.0-bin.tar.gz /usr/local/hive
2. Unpackage Hive

tar -xzvf apache-hive-2.0.0-bin.tar.gz
3. Set Hive environment variable

You will need to set the following in the environment

export HIVE_HOME=/usr/local/hive/apache-hive-2.0.0-bin
export PATH=$HIVE_HOME/bin:$PATH
So, you can do the following to export Hive variable to the environment when user log in

a. Create a  /etc/profile.d/hive.sh

sudo vi /etc/profile.d/hive.sh
b. Add the following in /etc/profile.d/hive.sh

export HIVE_HOME=/usr/local/hive/apache-hive-2.0.0-bin
export PATH=$HIVE_HOME/bin:$PATH
c. Source thi…

Linux - Locking a file

You could use the following command to lock a Linux file


#Open myfile as file descriptor 2
$ exec 2>myfile
#lock the file descriptor 2
$ flock -x 2
And release the lock with the following

$ exec 2>&-
Then, you could show file descriptor holding with the following command

lsof | grep myfile
or

lsof myfile
It could return something like

bash        132  userX    2wW   REG    8,2    0 135612 myfile
The following is the man page from lsof


FD
is the File Descriptor number of the file or:
cwd current working directory; Lnn library references (AIX); err FD information error (see NAME column); jld jail directory (FreeBSD); ltx shared library text (code and data); Mxx hex memory-mapped type number xx. m86 DOS Merge mapped file; mem memory-mapped file; mmap memory-mapped device; pd parent directory; rtd root directory; tr kernel trace file (OpenBSD); txt program text (code and data); v86 VP/ix mapped file;
FD is followed by one of these characters, describing the mode under which the…