Log File using system.out

Log File using system.out

If you want your System.out.print() output to be logged in the file rather than console, then that could also be possible by little trick.

OOP_SystemOutPrintln

Before we start,

System.out.print()
 :: System - Java Main Class
 :: out - Instance of PrintStream
 :: print() - A public Method


To achieve this we need to change the property of out using System-class.

System.setOut(PrintStream p);

PrintStream : The PrintStream class provides methods to write data to another stream. The PrintStream class automatically flushes the data so there is no need to call flush() method. Moreover, its methods don’t throw IOException.

Example:

package com.helical;

import java.io.File;
import java.io.FileNotFoundException;
import java.io.PrintStream;

public class SystemOutINFile {
    public static void main(String arr[]) throws FileNotFoundException
    {
        // Creating a log file object
        PrintStream printObject = new PrintStream(new File("log.txt"));

        // Storing current System.out in PrintStream before assigning some new value
        PrintStream console = System.out;

        // Assign printObject to output stream
        System.setOut(printObject);
        System.out.println("Whatever will be here in this function, it will go to text file");

       // Reassign the value for output stream
       System.setOut(console);
       System.out.println("console output");
    }
}


Hurray , That’s it….

Thanks for your time , Have a good Day 🙂

Pushpraj Kumar(Helical IT Solutions)

Advertisements

Change Hive metastore from derby to MySQL

Change Hive metastore from derby to MySQL

 

Machine : UBUNTU-14.04 | Hive : HIve 1.2.1

To change Hive Metastore from Derby to MySQL we need to follow these 8 simple steps,

 

Step-1 :

First we need to install Mysql and its dependencies on system.

Command-1 : sudo apt-get install mysql-server

Note: Click Next > Next and set the password for MySQL.

Command-2 : sudo apt-get install libmysql-java

 

Step-2 :

Create soft-link for connector in Hive lib directory.

Command-1 : ln -s /usr/share/java/mysql-connector-java.jar $HIVE_HOME/lib/mysql-connector-java.jar

 

Step-3 :

Access your MySQL and create one new database metastore for hive,

Command : $ mysql -u root -p

Enter password:

mysql> CREATE DATABASE metastore;

 

Step-4 :

Then we need one MySQL account for Hive to access the metastore. It is very important to prevent this user account from any type of change in schema.

Command :

mysql> CREATE USER ‘hive’@’%’ IDENTIFIED BY ‘password’;

mysql> GRANT all on *.* to ‘hive’@localhost identified by ‘password’;

mysql> flush privileges;

 

Step-5 :

Now we need to configure Hive to access MySQL metastore, for this we need to update hive-site.xml file(If file does not exist then create a new one)

<configuration>

<property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:mysql://192.168.8.99/metastore?createDatabaseIfNotExist=true</value>

<description>metadata is stored in a MySQL server</description>

</property>

<property>

<name>javax.jdo.option.ConnectionDriverName</name>

<value>com.mysql.jdbc.Driver</value>

<description>MySQL JDBC driver class</description>

</property>

<property>

<name>javax.jdo.option.ConnectionUserName</name>

<value>hive</value>

<description>user name for connecting to mysql server</description>

</property>

<property>

<name>javax.jdo.option.ConnectionPassword</name>

<value>password</value>

<description>password for connecting to mysql server</description>

</property>

</configuration>

 

Note: While updating please take all properties-tag only.

 

Step-6 :

Now we need to run the Hive schematool to initialization MySQL metastore.

For this we need to go to $HIVE_HOME>bin> folder

Command-1 : schematool -initSchema -dbType mysql

Note : When you have found that your metastore is corrupted, then we need to update metastore.

  • Before you run hive for the first time, run

Command : schematool -initSchema -dbType mysql

  • If you already ran hive and then tried to initSchema and if it’s failing:

Command : mv metastore_db metastore_db.tmp

You find your metasore_db file at $HIVE_HOME location.

  • Re run

 

Step-7 :

Start your Hive and access your tables.

 

Step-8 :

To validate it ,

Connect and open your hive

hive>

Then create a table in it and insert one record.

hive> create table saurzcode(id int, name string);

hive> insert into saurzcode(1, “Helical”);

Later access your MySQL and open metastore database

mysql -u root -p

Enter password:

mysql> use metastore;

And see your table as a record in TBLS table of metastore database.

mysql> show tables ;

mysql> select * from TBLS;

 

Hurray, Completed !!!!

Thanks for visiting , Have a great day.

PUSHPRAJ KUMAR

Collection Part-1 : Java Arrays

Arrays are objects which store multiple variables of the same type, it is a collection of similar type of elements that have contiguous memory location.

The length of an array is established when the array is created. After creation, its length is fixed.

array10No

An array of 10 elements.

It is an array of size 10, means we can store 10-elements in a single variable/object.

Define an array:

int[] arr = new int[10];

int[] arr ={1,2,3,4,5};

//We can give any number here. It will allocate 10 int-variable space in JVM heap section.

Starting with arr[0]=10;

to

arr[arr.length -1] =100; // arr.length -1=9 , because we are starting from 0 index.

How to copy one array content to another array,

public static void arraycopy( Object sourceArray, int sourceArrayPositionPos, Object destinationArray, int destinationArrayPosition, int length )

 

src — This is the source array.

srcPos — This is the starting position in the source array.

dest — This is the destination array.

destPos — This is the starting position in the destination data.

length — This is the number of array elements to be copied.

Use this function to copy one array into another . Ex: //here declaration, instantiation and initialization all are done at a time int[] arr ={8,7,6,5,4,3,2,2,0}; //Assume it is a sourceArray. int[] FinalArr = new int[9]; //Assume it is a destination array , where you want to copy your content. So copy command is:

System.arraycopy(arr, 0, FinalArr, 0, arr.length); //to print this destination/Targeted array.

for(int i=0;i<arr.length;i++)

{ System.out.println(“Value-”+i+” are : ”+FinalArr[i]); }

O/P

Value-0 are : 8

Value-1 are : 7

Value-2 are : 6

Value-3 are : 5

Value-4 are : 4

Value-5 are : 3

Value-6 are : 2

Value-7 are : 2

Value-8 are : 0

EX:

int c1[][][][]={{{{1,3,4},{3,4,5},{1,2,3}},{{1,3,4},{3,4,5},{1,2,3}},{{1,3,4},{3,4,5},{1,2,3}}},{{{1,3,4},{3,4,5},{1,2,3}},{{1,3,4},{3,4,5},{1,2,3}},{{1,3,4},{3,4,5},{1,2,3}}}};

for(int i=0;i<2;i++){

for(int j=0;j<3;j++){

for(int k=0;k<3;k++){

for(int l=0;l<3;l++){

System.out.print(c1[i][j][k][l]+””);

}

}

System.out.print(“\t”);

}

System.out.print(“\n”);

}

O/P

134345123 134345123 134345123
134345123 134345123 134345123

Enjoy learning…. 🙂

Batch-Updation in Hibernate

Batch Updation in Hibernate

JDBC has long been offering support for DML statement batching. By default, all statements are sent one after the other, each one in a separate network round-trip. Batching allows us to send multiple statements in one-shot, saving unnecessary socket stream flushing.

Hibernate hides the database statements behind a transactional write-behind abstraction layer. An intermediate layer allows us to hide the JDBC batching semantics from the persistence layer logic. This way, we can change the JDBC batching strategy without altering the data access code.

Update code snippet look like this ,

Session session = sessionFactory.openSession();
Transaction txInstance = session.beginTransaction();
ScrollableResults studentInstance = session.createQuery("FROM STUDENT").scroll();
int count =0;
while( studentInstance.next())
{
   Student student =(Student) studentInstance.get(Student.class,StudentID); 
   student.setregNo( regNO );
   session.update(student);
  // 50 - size of batch which you set earlier.
  // For Detail "http://helicaltech.com/batch-insertion-in-hibernate/"
   if(++count %50==0)
   {
      session.flush();
      session.clear();
   }
}
txInstance.commit();
session.close();

----------------------------------

 

PUSHPRAJ KUMAR (BI Developer)

Batch-Insertion in Hibernate

Batch-Insertion in Hibernate

To enable batch-processing to upload a large number of records into your database using Hibernate.

If you are undertaking batch processing you will need to enable the use of JDBC batching. This is absolutely essential if you want to achieve optimal performance. Set the JDBC batch size to a reasonable number (10-50, for example):

hibernate.jdbc.batch_size 20

the code snippet look like this ,

Session session = sessionFactory.openSession();
Transaction txInstance = session.beginTransaction();
   
for ( int i=0; i<100000; i++ ) {
    Student student = new Student(.....);
    session.save(student);
    if ( i % 40 == 0 ) { 
        session.flush();
        session.clear();
    }
}
   
txInstance.commit();
session.close();

When making new objects persistent flush() and then clear() the session regularly in order to control the size of the first-level cache because by default, Hibernate will cache all the persisted objects in the session-level cache and ultimately your application would fall over with an OutOfMemoryException.

A JDBC batch can target one table only, so every new DML statement targeting a different table ends up the current batch and initiates a new one. Mixing different table statements is therefore undesirable when using SQL batch processing.

PUSHPRAJ KUMAR (BI Developer)

 

JAVA POLICY FILE..


JAVA POLICY FILE Apps-File-Java-icon

 

 

The Java™ 2 Platform, Enterprise Edition (J2EE) Version 1.3 and later specifications have a well-defined programming model of responsibilities between the container providers and the application code.

 
The java.policy file is a global default policy file that is shared by all of the Java programs that run in the Java virtual machine (JVM) on the node. A change to the java.policy file is local for the node.

 
The java.policy file is not a configuration file that is managed by the repository and the file replication service. Changes to this file are local and do not get replicated to the other machine.
By using this feature we can control the execution ,
To set the run-time permissions such that Java won’t grant the global permissions. Then you can specify only the permissions you want granted for your app. The key is to run your app with the options below.

java -Djava.security.manager -Djava.security.policy==policyFile.txt MyClass

Note the double equals -Djava.security.policy==policyFile.txt. The double equals == means to use only the permissions in the named file as opposed to the single equal sign -Djava.security.policy=policyFile.txt which means use these permissions in addition to the inherited global permissions.

 
Then create a policy file excluding the permissions you want to deny:

//policyFile.txt
grant codeBase “file:/C:/abc.jar” {
//list of permissions minus the ones you want to deny
//for example , the following give the application
//ONLY AudioPermission and AWTPermission. Other permission such as
//java.io.FilePermission would be denied.
permission javax.sound.sampled.AudioPermission;
permission java.awt.AWTPermission;
}

NOTE : {app_server_root}/java/jre/lib/security/java.policy. Default permissions are granted to all classes. The policy of this file applies to all the processes launched by `Application Server.

 

 

Pushpraj Kumar

Begining with JNI

                                                 Java Native Interface (JNI)    jni

Java provides a framework called the Java Native Interface (JNI), with which it is possible to write native methods. A native method is a method that is mapped to a function in a library of native code, such as a DLL under Windows. Or, to put things more simply, you can write code in C or C++1 or any other language and call it from Java.

Purpose and features

JNI enables programmers to write native methods to handle situations when an application cannot be written entirely in the Java programming language, e.g. when the standard Java class library does not support the platform-specific features or program library. It is also used to modify an existing application—written in another programming language—to be accessible to Java applications. Many of the standard library classes depend on JNI to provide functionality to the developer and the user, e.g. file I/O and sound capabilities. Including performance- and platform-sensitive API implementations in the standard library allows all Java applications to access this functionality in a safe and platform-independent manner.

The JNI framework lets a native method use Java objects in the same way that Java code uses these objects. A native method can create Java objects and then inspect and use these objects to perform its tasks. A native method can also inspect and use objects created by Java application code.

JNI is sometimes referred to as the “escape hatch” for Java developers because it enables them to add functionality to their Java application that the standard Java APIs cannot otherwise provide. It can be used to interface with code written in other languages, such as C and C++. It is also used for time-critical calculations or operations like solving complicated mathematical equations, because for some classes of problems native code may run faster than JVM code

How the JNI works

In the JNI framework, native functions are implemented in separate .c or .cpp files. (C++ provides a slightly simpler interface with JNI.) When the JVM invokes the function, it passes a JNIEnv pointer, a jobject pointer, and any Java arguments declared by the Java method. A JNI function may look like this:

 

JNIEXPORT void JNICALL Java_ClassName_MethodName(JNIEnv *env, jobject obj)

{

/*Implement Native Method Here*/

}

 

The env pointer is a structure that contains the interface to the JVM. It includes all of the functions necessary to interact with the JVM and to work with Java objects. Example JNI functions are converting native arrays to/from Java arrays, converting native strings to/from Java strings, instantiating objects, throwing exceptions, etc. Basically, anything that Java code can do can be done using JNIEnv, albeit with considerably less ease.

 

The argument obj is a reference to the Java object inside which this native method has been declared.

 

For example, the following converts a Java string to a native string:

 

//C++ code

extern “C”

JNIEXPORT void JNICALL Java_ClassName_MethodName(JNIEnv *env, jobject obj, jstring javaString)

{

//Get the native string from javaString

const char *nativeString = env->GetStringUTFChars(javaString, 0);

//Do something with the nativeString

//DON’T FORGET THIS LINE!!!

env->ReleaseStringUTFChars(javaString, nativeString);

}

 

/*C code*/

JNIEXPORT void JNICALL Java_ClassName_MethodName  (JNIEnv *env, jobject obj, jstring javaString)

{

/*Get the native string from javaString*/

const char *nativeString = (*env)->GetStringUTFChars(env, javaString, 0);

 

/*Do something with the nativeString*/

 

/*DON’T FORGET THIS LINE!!!*/

(*env)->ReleaseStringUTFChars(env, javaString, nativeString);

}

 

/*Objective-C code*/

JNIEXPORT void JNICALL Java_ClassName_MethodName(JNIEnv *env, jobject obj, jstring javaString)

{

/*DON’T FORGET THIS LINE!!!*/

JNF_COCOA_ENTER(env);

/*Get the native string from javaString*/

NSString* nativeString = JNFJavaToNSString(env, javaString);

/*Do something with the nativeString*/

/*DON’T FORGET THIS LINE!!!*/

JNF_COCOA_EXIT(env);

}

Native data types can be mapped to/from Java data types. For compound types such as objects, arrays and strings the native code must explicitly convert the data by calling methods in the JNIEnv.

JNI and OSGi

The following graphic shows the Java Native Interface (JNI) that allows native code (C or C++) to interact with bytecode. The OSGi Framework is a dynamic module system that provides a way to deploy a new version of a software module without having to reboot the container of the modules. One use case is to automate updates for the industrial internet, such as:

  • telecommunications devices that have many modules (camera, video player, audio recording, audio player, web browser, text input, text display), or
  • devices that serve medical purposes. For example, a CAT-scanner uses software to gather and to analyze x-ray data. A new version of the analytical software could be provided without having to shut down the acquisition system. Another use case involves the separation of the database layer from the business rules layer from the view layer. (Compare to the Model-View-Controller pattern.)
  • osgi

Pushpraj Kumar….