Working on remote linux server

Logging into remote linux server:

$ ssh username@remoteserver.com

Now enter your password and voila :D. Probably you know this. Just warming up a bit.

» What if ssh server is opened at a different port other than default 22?
If you server opened ssh in a different port, you can use this.

$ ssh -p 9009 username@remoteserver.com

» Want to transfer a war file to remote server?
$ scp demo.war username@remoteserver:/opt/apache/tomcat/webapps/
Or copying log files back to your machine?
$ scp -r username@remoteserver:/opt/apache/tomcat/logs /home/myself/

» You may not want to use password every time you log into the server.

You can create public-private key pair on you machine and share the public key with the remote server for ssh to use it.
Create key pair on your machine:
$ ssh-keygen -f ~/.ssh/id_fun
You'll find two files in ~/.ssh folder id_fun and id_fun.pub which are the private and public key respectively.
share id_fun.pub with the remote server and add to ~/.ssh/authorized_keys files
$ scp ~/.ssh/id_fun.pub username@remoteserver:/home/username/.ssh/
$ ssh username@remoteserver
# you need to enter password now
username@remoteserver$ cat ~/.ssh/id_fun.pub >> authorized_keys

Go back to you machine and try 'ssh username@remoteserver' and log into remote server without any password.
Do i have to tell you that you don't share id_fun with any one.

» Didn't like scp'ing. Why not mounting remote server's file system to yours?
$ mkdir /mnt/serverishere
$ sshfs username@remoteserver:/home/username/eveything /mnt/serverishere
Now you can find everything folder is mounted to /mnt/serverishere on you machine.

Technorati Tags: linux, shell command, ssh, remote server, ssh key,

Monitor you Java application remotely

Perhaps you used jconsole or newer jvisualvm to monitor Java application on your machine.

Or, may be you monitored your tomcat server remotely using this type of URI in jconsole

'service:jmx:rmi:///jndi/rmi://server:9024/jmxrmi'

Now, its time to do the same for the Java application you have written.

The idea to expose your platform MBean server through RMI

// get the MBean server.

MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();

// expose the MBean server through a service URL.

JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:9999/server");

JMXConnectorServer cs = JMXConnectorServerFactory.newJMXConnectorServer(url, null,mbs);

cs.start();

Ohh.. you have to open an RMI registry at port 9999, for sure.

rmiregistry 9999

Now, what are the benefits are you having monitoring remotely?

- First of all, using jconsole, jvisualvm can be a big performance issue for the application you are running on the same machine. Moreover, you production server is an Ubuntu server, so you cannot run Jconsole.

JDK useful commands and tools

jps - see all the PID's of all Java processes running on the machine

$ jps

11767 Jps

6854 Main

8205 Bootstrap

jstat - See how many classes are loaded by JVM for a Java program

$ jstat -class [pid]

Loaded Bytes Unloaded Bytes Time

10271 12074.2 0 0.0 11.10

jstack - Create thread dump

$ jstack [pid]

Useful usage of jmap

See summary of the heap memory

$ jmap -heap [pid]

Dump a heap file

$ jmap -dump:binary=b,file=[filename.dump] [pid]

Dump a heap file with only live objects

$ jmap -dump:live,binary=b,file=

Number of objects of a specific type

$ jmap -histo [pid] | grep org.example.MyClass

jinfo - To see system properties and VM flags

$ jinfo [pid]

Visual tools

jconsole - Graphical management console to see a number of attributes and graphs of a Java program. Simple usage:

$ jconsole [pid]

jvisualvm - This is newer than jconsole. Thread dump and heap dump creator and profiler for heap dump. This is also a monitoring tool like jconsole.

jhat - See the heap created. (probably this is not a very useful one. Some profiler/jvisualvm can be used)

$ jhat -port 8081 now.heap

Google Gears - features for performance web application

What does web application gets from Google Gears?
Straight and simple as google says..
- Interect with Desktop
- Use gears local server for caching.
- Store data in offline mode and synchronize when online.

Last thing is seems most interesting to me.. I can use the webpages even if i am not connected to internet. Thats cool. I used an webapp that uses google gears http://www.rememberthemilk.com for this purpose.


[ Lightweight database on client Side ]
Gears uses SQLite for database support. And its secured, because a web application cannot access data outside of its domain. More importantly it supports fulltext search.
Using this Database API is very Simple Only two classes Database and ResultSet. Using this api is so simple i cant help copying the code. :D

// db is instance of Database
var db = google.gears.factory.create('beta.database');
db.open('my-database');
// rs is Instance of ResultSet
var rs = db.execute('select * from TestTable order by Timestamp desc');
while (rs.isValidRow()) {
alert(rs.field(0) + '@' + rs.field(1));
rs.next();
}
Its simple :). This feature is actually for storing user data in offline mode and when user is online, this information has to be synchronized. Still i am not clear enough about how synchronization should be done.

[ Another most interesting feature : LocalServer. ]
Pages and files/resources can be served in offline mode from Localserver. When localServer is configured to serve some url, it caches those files/resources with urls. Each time that url is requested localserver servers those resources/files without sending actual server request. Thats one cool thing.
How does Localserver know which urls to cache?A manifest file is used to tell the urls and version info of the urls. Local server gets the manifest file each time. And update resource from urls if it doesn't have the latest version.

[ Using Google Gears in your page. ]
* Install Google gears first http://gears.google.com/
* Download this script gear_init.js
* Include this script in the page

=: First thing you should in your page is check whether this pc has google gears installed.
if (!window.google) {
alert("gear is not here!");
location.href = "http://gears.google.com/?action=install&return=http://yoursite.com";
}

=: How do you get the api implementation of Gears? Gears provides an Factory for this.
This is how you will get the LocalServer to cache files.
var localServer = google.gears.factory.create("beta.localserver");
And get the database..
var databae = google.gears.factory.create("beta.database");

Using these localserver and database is simple and i am not going to present them... google documented them good enough.


[ Other supports from Gears]
There are other supports like httpRequest, Timer. But they are not new in Javascript programming. Actually these are implemented to be used from Gears WorkerPool

For "Interect with Desktop" They should have said file access feature..
Because their implementation gives two feature.
- Can create shortcut in pc file system.
- get a file content from the pc files system. File content is served in Blob format.

Oh here is one thing new here "Blob". That's an implementation by Google Gear.
Javascript itself doesn't have Blob support. Blob can be passed into, and returned by, a variety of Gears methods.

Other cool things in Google Gears are GeoLocation & Workerpool API. Hope to give a demonstration on those soon..

putting debian dependency while creating .deb by alien

A .rpm package can be converted to .deb package using following command
sudo alien --to-deb --scripts someone-0.11-4.i386.rpm
it will generate a .deb package someone_0.11-5_i386.deb

What alien cannot resolve is converting rpm dependencies to debian dependencies..

So, here is a solution.. (there might be others).. putting the dependencies manually..

1. convert the rpm package to debian package format directory. use command
sudo alien --generate --scripts ecedemo-0.11-1.noarch.rpm
this will create directory someone-0.11/ with deb package like structure.

2. now, all you have to do is change the someone-0.11/debian/control file. add whatever dependencies you like in the for "Depends: " tag.
Depends:sun-java5-jre, slapd

3. rebuild deb package from intermediate direcotory
cd someone-0.11/
sudo dpkg-buildpackage

Logging with log4j : Options and Features

Here are some tricks and options how log4j logger can be configured.
If you are new to Apache Log4j, you should consider this post first... logging-basics-with-log4j

Separating logger.

You may need different log output behavior from different classes.
Suppose your application uses Hibernate and you want all hibernate-generated logs to go in a special file, say "app.hibernate.log". So, what you need is to define separate logger configuration for Hibernate classes.

log4j.rootLogger=DEBUG, A1, F1
...
..
log4j.logger.org.hibernate=DEBUG, H1
log4j.appender.H1=org.apache.log4j.FileAppender
log4j.appender.H1.File=logs/app.hibernate.log
....
..

That is Logger of class org.hibernate.Session , Logger class of org.hibernate.criterion.DetachedCriteria will use the configuration for log4j.logger.org.hibernate.

One more trick... if you don't want to include the hibernate log to be added in root logger, add this in config.
log4j.additivity.org.hibernate=false

Level inheritance of loggers

If logger's level is not specified, it will inherit it's parent logger's level.
if you specify org.hibernate logger like this log4j.logger.org.hibernate will inherit level of log4j.rootLogger
log4j.logger.org.hibernate=, H1
or
log4j.logger.org.hibernate=INHERITED, H1

What is appender?

If logger's level is not specified in the configuration it will inherit it's parent logger's level.

Appender is a component in log4j. Appender decides

• where to put your log output ( file, console, ..)
  log4j.appender.C1=org.apache.log4j.ConsoleAppender
  
• what is the format of your log.
  log4j.appender.C1.layout=org.apache.log4j.PatternLayout
  
• to which level log output should be filtered.
  log4j.appender.E1.threshold=ERROR

You can specify these things in logger configuration.
Mind it, A logger can have multiple appenders.

Using file appender

File Appenders are used to store log output in files. There are several file appenders in log4j with different file creation policy and layout . Configuration of file appenders includes file name, file creation policy, output pattern. Here are some common issues with file appenders.

When you don't wanna loose previous log

By default, Every time you restart your program, your previous log file is removed by new log. But if you want your previous log to persist and new logs to be appended in that file, use following configuration.
log4j.appender.F1.Append=true

Manage fixed size log files.

Hmm.. you are not loosing any log, but aren't you creating enormous log file. Don't worry. You can configure appender to split log file when they grow upto a predefined size. For this, you have to use RollingFileAppender instead of FileAppender and define the threshold size of the log file.

log4j.appender.X1=org.apache.log4j.RollingFileAppender
log4j.appender.X1.MaxFileSize=10240KB
log4j.appender.X1.MaxBackupIndex=12
This configuration will create log files of at most 10MB and will not create more than 12 log files.

Manage daily log files

You want your server to create and use new log file every day. Try DailyRollingFileAppender
log4j.appender.F1=org.apache.log4j.DailyRollingFileAppender
log4j.appender.F1.File=logs/server_daily.log
log4j.appender.F1.DatePattern='_'yyyy-MM-dd'.log'

Send mail for error logs

You want your application to notify you when it puts a log of FATAL level? ( you can do it for other levels too). You don't need to write code for sending mail. You can configure SMTPAppender to do this for you.
log4j.appender.E2=org.apache.log4j.net.SMTPAppender
log4j.appender.E2.layout=org.apache.log4j.PatternLayout
log4j.appender.E2.layout.ConversionPattern=%d %-5p [%t] %-17c{2} (%13F:%L) %3x - %m%n
log4j.appender.E2.SMTPHost=exchange.spectrum-bd.com
log4j.appender.E2.To=rkzico@spectrum-bd.com, ruby@spectrum-bd.com
log4j.appender.E2.From=error@spectrum-bd.com
log4j.appender.E2.threshold=ERROR

Appender Layouts

Other than pattern layout There are some nice Layouts.
An appender with HTMLLayout lets you see your log output in web browser in a table format.
sample.
log4j.rootLogger=DEBUG, htm
...
log4j.appender.htm.layout=org.apache.log4j.HTMLLayout
..

You can have your log to be saved as XML data by using XMLLayout. You can make intelligent use of this XML log information and also see log files in tools like Apache chainsaw.

Logging Basics with Log4j

Why use a smart logging tool...

• We can put log output to different presentations like console, log file.
• We can categorize log entries to several level of importance and purpose (Debug/ Info/ Error). And then restrict log output to a certain level when we want.
• Logger provides time, class, other information in log output.
• You can personalize the format of log output.
  
• And many other…

How to put log4j logging in my code…

• Configuration: Before using Logger class for logging, you need to load configuraion in your program. Otherwise your log will not be printed or put anywhere.
  
  
• write a config file "myApp.log.config". Following is a simple configuration to show log in console and store in a file "test.log"
  

  log4j.rootLogger=DEBUG, A1, F1
  

  log4j.appender.A1=org.apache.log4j.ConsoleAppender
  

  log4j.appender.A1.layout=org.apache.log4j.PatternLayout

  log4j.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x -%m%n

  log4j.appender.F1=org.apache.log4j.FileAppender

  log4j.appender.F1.File=${user.home}/test.log

  log4j.appender.F1.layout=org.apache.log4j.PatternLayout

  log4j.appender.F1.layout.ConversionPattern=%p %t %c - %m%n
  details of configuration...
  

• Load the configuration from code
  

  PropertyConfigurator.configure("myApp.log.cofig");
  
  

• Log Now! You are ready to log.
  
  • Get the Logger class
    
  Logger logger = Logger.getLogger( this.getClass() );
  Logger logger = Logger.getLogger( MyClass.class );
  Logger logger = Logger.getLogger( "ClassName" );
  
  
  • Log your message in appropriate level
    
  logger.debug ( "No of content loaded : " + lenContentList );
  logger.error( "Error occurred while parsing", execption );
  logger.fatal( "Failed to connect the server" );
  

Appropriate use of logging levels

Here is a guideline for deciding the logging level when you put a message in the log.

• FATAL: at which point your application or part of application is no longer functional to serve.
• Crash of the application.
• Crash the relevant sub-component.

• ERROR
• Usually when a Java exception occurs.
  
• Processing of a request is postponed due to some error.
• Processing a request has not cover all features.

• WARN: indicates minor problems
• Factors external to the application. Such as missing or inconsistent input parameters supplied by user.
• Improper initialization, configuration etc.

• INFO:
  
  • Significant non-error events in the normal life cycle of the application.
  • When a request is successfully processed in your application.

• DEBUG
  
• Printing value of a variable or object property in log .
  
• Non-error events in the normal life cycle of the application

• TRACE
• Usually used for timing information.