Tuesday, May 2, 2017

Java SE 9... What's New?

I have provided some of the most important core language enhancements for JDK 9.0, along with code samples. The examples provided below can be directly pasted into your IDE and you may name the class as provided.

JEP 261: Module System Module System in JDK 9
JEP 200: The Modular JDK Module System in JDK 9
JEP 220: Modular Run-Time Images Module System in JDK 9
JEP 260: Encapsulate Most Internal APIs Module System in JDK 9
JEP 223: New Version-String Scheme Changes in JDK 9
Enable or Disable Web Deployment with Installer's UI JDK 9 Installer
JEP 158: Unified JVM Logging Tools in JDK 9
JEP 214: Remove GC Combinations Deprecated in JDK 8 Tools in JDK 9
JEP 222: jshell: The Java Shell (Read-Eval-Print Loop) Tools in JDK 9
JEP 224: HTML5 Javadoc Tools in JDK 9
JEP 228: Add More Diagnostic Commands Tools in JDK 9
JEP 231: Remove Launch-Time JRE Version Selection Tools in JDK 9
JEP 240: Remove the JVM TI hprof Agent Tools in JDK 9
JEP 241: Remove the jhat Tool Tools in JDK 9
JEP 245: Validate JVM Command-Line Flag Arguments Tools in JDK 9
JEP 247: Compile for Older Platform Versions Tools in JDK 9
JEP 282: jlink: The Java Linker Tools in JDK 9
JEP 219: Datagram Transport Layer Security (DTLS) Security in JDK 9
JEP 244: TLS Application-Layer Protocol Negotiation Extension Security in JDK 9
JEP 249: OCSP Stapling for TLS Security in JDK 9
JEP 246: Leverage CPU Instructions for GHASH and RSA Security in JDK 9
JEP 273: DRBG-Based SecureRandom Implementations Security in JDK 9
JEP 229: Create PKCS12 Keystores by Default Security in JDK 9
JEP 287: SHA-3 Hash Algorithms Security in JDK 9
Deprecate the Java Plug-in Deployment in JDK 9
Enhanced Java Control Panel Deployment in JDK 9
JEP 275: Modular Java Application Packaging Deployment in JDK 9
JEP 289: Deprecate the Applet API Deployment in JDK 9
JEP 213: Milling Project Coin Java Language in JDK 9
JEP 221: Simplified Doclet API Javadoc in JDK 9
JEP 224: HTML5 Javadoc Javadoc in JDK 9
JEP 225: Javadoc Search Javadoc in JDK 9
JEP 261: Module System Javadoc in JDK 9
JEP 165: Compiler Control VM in JDK 9
JEP 197: Segmented Code Cache VM in JDK 9
JEP 276: Dynamic Linking of Language-Defined Object Models VM in JDK 9
JEP 271: Unified GC Logging JVM Tuning in JDK 9
JEP 248: Make G1 the Default Garbage Collector JVM Tuning in JDK 9
JEP 102: Process API Updates Core Libraries in JDK 9
JEP 193: Variable Handles Core Libraries in JDK 9
JEP 254: Compact Strings Core Libraries in JDK 9
JEP 264: Platform Logging API and Service Core Libraries in JDK 9
JEP 266: More Concurrency Updates Core Libraries in JDK 9
JEP 268: XML Catalogs Core Libraries in JDK 9
JEP 269: Convenience Factory Methods for Collections Core Libraries in JDK 9
JEP 274: Enhanced Method Handles Core Libraries in JDK 9
JEP 277: Enhanced Deprecation Core Libraries in JDK 9
JEP 285: Spin-Wait Hints Core Libraries in JDK 9
JEP 290: Filter Incoming Serialization Data Core Libraries in JDK 9
JEP 259: Stack-Walking API Core Libraries in JDK 9
JEP 236: Parser API for Nashorn Nashorn in JDK 9
JEP 292: Implement Selected ECMAScript 6 Features in Nashorn Nashorn in JDK 9
JEP 251: Multi-Resolution Images Client Technologies in JDK 9
JEP 256: BeanInfo Annotations Client Technologies in JDK 9
JEP 262: TIFF Image I/O Client Technologies in JDK 9
JEP 263: HiDPI Graphics on Windows and Linux Client Technologies in JDK 9
JEP 272: Platform-Specific Desktop Features Client Technologies in JDK 9
JEP 283: Enable GTK 3 on Linux Client Technologies in JDK 9
JEP 267: Unicode 8.0 Internationalization in JDK 9
JEP 252: CLDR Locale Data Enabled by Default Internationalization in JDK 9
JEP 226: UTF-8 Properties Files Internationalization in JDK 9


As a Developer, the ones that impact our day-to-day lives mostly fall in the areas of:

1. Java Language in JDK 9
2. Core Libraries in JDK 9
3. Changes in JDK 9

Even though others are also very important, they can be studied, researched and mastered as and when the need arises. Before we get into the Java 9 Changes, let us get into the Java 8 changes that matter to understand each of these better. I will only enlist those changes that I have not covered in my earlier blog entry at: http://techilashots.blogspot.in/2015/05/java-se-8-whats-new-part-0404.html 

1. Default Interface Methods
Whenever there is existing or legacy code which has Interfaces that require addition of new methods - It causes breakage of existing classes that inherit/implement from this Interface unless the implementation for each of these added methods is provided in the classes. This is not very maintainable code. Even though a good practice as per SOLID and other OO paradigms to provide an interface without any implementation - we need to handle and solve the problem as mentioned above. This is where Default Interface Methods come in. 

 import java.util.List;  
3:  public interface LegacyPublicInterface {  
6:      /**  
7:       * Additional Default Method that Can be Invoked on Each Type of Invoice that Implements the LegacyPublicInterface.   
8:       * It can also be over-ridden by the Extending Invoice Types. This is an Example Usage and Benefit of the Java/JDK 8  
9:       * Default Interface feature.  
10:       * @param items  
11:       */  
12:      default void checkStatusOnEachItem (List<String>... items) {  
14:          for(String item : items) {  
15:              if(item.startsWith("OOS_")) {  
16:                  items.remove(item);  
17:              }  
18:          }  
19:          return;  
20:      }  
22:  }  

From the Example Above, the LegacyPublicInterface in an Existing Application is already extended by Multiple Invoice Types (For Example in an Inventory System). But as per changing business, Re-Engineering effort requires that each of the Invoices have a method to invalidate or remove an Item marked with "OOS". Given such a problem, Prior to Java 8, we would have to introduce a new method declaration in the interface and then require that each of the implementing classes implement their own logic for handling this. With Default Interfaces, the task becomes very simple (The code is now more maintainable and extensible and requires very less effort to change). With the introduction of thus feature of Default Methods, the following are the possibilities:

1. Use the Default Method(s) without Breaking Existing Functionality (Best Use)
2. Implementing Class can choose to override these Default Methods
3. Abstract Classes can be provided over the Interfaces to override Implementation

So, Let's get further into each of these changes in Java 9, one-by-one.

1. Java Language in JDK 9

JEP 213: Milling Project Coin

[A] @SafeVarargs is Allowed on Private Instance Methods
Using non-reifiable variable argument parameters in a method can cause multiple warnings when trying to compile code, such as:

Note: LegacyPublicInterface.java uses unchecked or unsafe operations.  
Note: Recompile with -Xlint:unchecked for details.

Hence @SafeVarargs was introduced to suppress such warnings on unchecked or unsafe operations. By using this, the developer is signalling to the compiler that he has made sure that there will be no Heap Pollution (such as unsafe forEach operations) caused.

Prior to Java 9, @SafeVarargs is allowed on non-overridable methods such as in static methods, final instance methods and constructors. Note that the annotation will throw an error if it is used in fixed arity methods. 

In Java 9, @SafeVarargs can be used on private instance methods.

[B] Underscore as a Variable Name is not Legal Anymore
Using only the _ (underscore character) as a variable name is not legal anymore.This is because it is marked as a reserved keyword from Java 1.8 (But causing compilation failure only in Java 1.9).
This may cause a some issues when compiling legacy source code, especially which had a necessity to denote some specific resource or entity using the _ (underscore).  It may have to be rewritten and may have many related ramifications.

[C] Private Interface Methods have been Introduced 
Interfaces in Java 9 are allowed to have Private Methods. This was done to allow code sharing between non-abstract methods in the Interface. All rules related to ordinary Private modifier apply to these methods. The point to note is that a method cannot be both private and abstract. It definitely needs to have a method body.

[D] Allow Effectively final variables to be used as Resources in Try with Resources
Upto Java 8, Every variable that had to be used within Try with Resources statements requires to be declared within the try statement. Only then, can it be used within the try block. This is a limitation for th developer. Hence, In Java 9, this restriction has been removed and any final variable or effectively final (local) variable can be used inside the try block. All other rules as applicable to Try with Resources continue. Effectively Final means the variable that is not changed once after it has been initialized.

[E] Allow Diamond with Anonymous Classes If the Inferred Type's Argument Type is Denotable
Upto Java 8, using Generics and Diamond Operators with Anonymous Classes. It was mainly because the compiler could not infer whether it can represent the type in the Argument passed to the Diamond Operator.  JSR 334 has the following to say about using diamond with anonymous classes:
"Using diamond with anonymous inner classes is not supported since doing so in general would require extensions to the class file signature attribute to represent non-denotable types, a de facto JVM Change." 
Additional information is located in the Project Coin mailing list's Diamond Operator and Anonymous Classes topic:
"Internally, a Java compiler operates over a richer set of types than those that can be written down explicitly in a Java program. The compiler-internal types which cannot be written in a Java program are called non-denotable types. Non-denotable types can occur as the result of the inference used by diamond. Therefore, using diamond with anonymous inner classes is not supported since doing so in general would require extensions to the class file signature attribute to represent non-denotable types, a de facto JVM change. It is feasible that future platform versions could allow use of diamond when creating an anonymous inner class as long as the inferred type was denotable."
With Java 9, the Java Compiler has changed its Inference Algorithm in a way that Diamond Operator (Generics) can now work simultaneously with Anonymous Classes, as long as the Argument Type of the Inferred Type is Denotable. The important point to note is that things that fall under Denotable are Primitive Types, Raw Types and Non-Generic Types. Non-Denotable means ones that cannot be written in a Java Program, like usage of Extends, Super along with Wildcard Types in Generics. These are usually inferred by the compiler. So, as long as the compiler identifies that the Argument Type of Inferred Type is Denotable - You can use the Diamond Operator in Conjunction with Anonymous Inner Classes.

2. Core Libraries in JDK 9

With Java 9, one can retrieve the PID of the process through a native call. This is achievable through the ProcessHandle. Also, we can retrieve information about the currently running Java Process (JVM) and Info (Inner Class of ProcessHandle) Class/Object that contains details on the process. We can also enlist or return a snapshot of all currently running processes in the system.

Java's Concurrent Package (java.util.concurrent.atomic) provide all Atomic Types for performing atomic operations. Apart from this,  Unsafe Operations (sun.misc.unsafe) such as creating objects without calling the constructor used in Java Low-Level Programming require to be hidden from the outside world as per JEP 260: Encapsulate Most Internal APIs This has led to creation of a new abstract class type named VarHandle - This will allow a developer to assign different types to the same reference (dynamically typed reference). It can also take care of performing atomic operations on the held variable, including compare and swap (set or exchange) operations. It also provides memory fencing operations, to order the in-memory representation of the object, by providing finer grain control.
Although this has no external ramification to a developer in terms of syntax or semantics change - It may impact the way we design for memory and performance. The current UTF-16 representation uses 2 Bytes for Storage. Most of the string contain characters that are only Latin-1 in nature. The Latin-1 characters require only 1 Byte for Storage. With Java 9, String storage has been modified to start with an Additional Encoding Flag. This flag indicates whether it contains ISO-8859-1/Latin-1 characters or the UTF-16 characters. As per the official word, It has lead to an Improved Usage of Memory and Efficient GC, but with Some Loss in Performance at Peak Loads.

JEP 264: Platform Logging API and Service
This defines a Minimal Logging API for the Platform and also provides a Service Interface for the consumers of these messages. The implementation for this interface can be provided by logging libraries or application itself to Route the Messages appropriately to the application specific logging framework or implementation being used such as [Log4J or SLF4J]. Whenever an implementation is not available, the run-time switches to Default Java Logging Package. (java.logging). This also allows to detect BootStrap Issues.

JEP 266: More Concurrency Updates
There is a continually evolving thought process on concurrency. With Java 9, An Interoperable Publish-Subscribe Framework has been provided. Also, enhancements to the CompletableFuture object has been provided, along with Numerous Implementation Improvements since JDK 8 that includes JavaDoc Rewording. An Interoperable Publish-Subscribe Framework, also known as Reactive Streams allow two-way communication based on the traffic or volume at each end of the stream, Publisher and Subscriber. The enhancements to the CompletableFuture API include methods that allow a Future to complete with a value or with exception, after a Timeout period. Also, a Delayed Executor has been provided to allow a task to execute after some delay.

JEP 268: XML Catalogs
An XML catalog is made up of entries from one or more Catalog entry files. A Catalog entry file is an XML file whose document element is Catalog and whose content follows the XML Catalog DTD defined by OASIS. With Java 9, a Public API for XML Catalog Management. The external references in the catalog needed to be resolved repetitively. But with this new API, an API containing CatalogManager, Catalog and CatalogResolver can be used to reduce or eliminate this external invocations, when the catalog resource already exists locally. It creates a Local Catalog and will allow JAXP based processors to resolve to this local catalog.

JEP 269: Convenience Factory Methods for Collections
This addition makes it convenient for the Developer to create Immutable Collections out of existing collections, be it Set, Map or List. A static factory method of() added to each of the interfaces - Set, Map and List. It is important you understand the following (even though consistent with previous versions of Java):
  • They are Structurally Immutable
  • They Disallow Null elements or null keys. 
  • They are Serializable if all elements are serializable.
  • They Reject Duplicate Elements/Keys at creation time. 
  • The Iteration Order of set elements is Unspecified and is Subject to Change.
  • They are Value-Based. Factories are free to create new instances or reuse existing ones. Therefore, Identity-Sensitive Operations on these instances (Reference Equality (==), Identity Hash Code, and Synchronization) are Unreliable and should be Avoided.They are serialized as specified on the Serialized Form page.

JEP 274: Enhanced Method Handles 
A method handle is a typed, directly executable reference to an underlying method, constructor, field, or similar low-level operation, with optional transformations of arguments or return values. These transformations are quite general, and include such patterns as conversion, insertion, deletion, and substitution [Incomplete Description]

JEP 277: Enhanced Deprecation
With an eye on maintainable and more informative code, the Developer defined Deprecation now allows us to mark deprecation with additional elements of information like forRemoval and since. The forRemoval allows allows to mark that this item may be removed in the future versions of Java and since provides information about when it was first introducted.

JEP 285: Spin-Wait Hints
For multi-threading applications, this brings in some performance improvements under Busy-Waiting or Spin-Waiting conditions. Usually, Busy-Waiting is done for synchronization of some state of the object between two or more invokers - Waiting for a condition to occur before processing starts or continues. Thread.onSpinWait() has been introduced as a static method in the Thread class and can be optionally called in Busy-Waiting loops - This will allow the JVM to issue processor instructions on some system architectures to improve Reaction Time in such Spin-Wait Loops and also Reduce the Power Consumed by the Core Thread or Hardware thread. This benefits the Overall Power Consumption of a Program, and possibly Allowing other Cores or Hardware Threads to execute at Faster Speeds within the Same Power Consumption Envelope.

JEP 290: Filter Incoming Serialization Data 
This feature is related to the addition of filters at the serialization incoming streams to improve security and robustness. The core mechanism is a filter interface implemented by serialization clients and set on an ObjectInputStream. The filter interface methods are called during the deserialization process to validate the classes being deserialized, the sizes of arrays being created, and metrics describing stream length, stream depth, and number of references as the stream is being decoded. The filter returns a status to accept, reject, or leave the status undecided.

JEP 259: Stack-Walking API 
Prior to Java 9, The way to access Stack Trace was very limited and provided the entire dump or stack information at once. This was inefficient and did not allow any direct way of filtering data. With Java 9, a Lazy StackWalker API has been introduced. This will allow to fetch data based on filtering conditions and is more efficient.

3. Changes in JDK 9 

JEP 223: New Version-String Scheme 
[Incomplete Description]

I will keep adding the below mentioned to this article. Please do check back again around 30-May-2017! :-)

1. Detailed Code Samples for All of the Above.
2. JEP 261: Module System 
3. JEP 200: The Modular JDK 
4. JEP 220: Modular Run-Time Images 
5. JEP 260: Encapsulate Most Internal APIs 


Happy Java with JDK9!

Sunday, April 23, 2017

Evolution of the Java Memory Architecture [Java 1.7 and Java 1.8]

Java Memory Architecture [Java Memory Architecture Before Java 7]
You will have to refer to my detailed write-up on this - It explains the core concepts of the Java Memory Architecture, the ones that should get you started to understand this evolution better. The article is at : https://dzone.com/articles/java-memory-architecture-model-garbage-collection Most of the original concepts of the Java Memory Architecture are covered in detail in this article. This articles also provides information on Garbage Collection, Memory Leaks and  Most Common Memory Out of Memory Issues.

String Literals in Java are stored in String Pool. String Interning refers to a process or method by which only one copy of a specific string is stored in the memory. This is is done to allow efficient usage of memory and also of less time to retrieve, except when the string is first created. This immutable single copy of the String is called intern. Java provides a method in the String class, intern() to actually create/retrieve this copy of the string.  

Pre-Java 7 Era (String Handling)

Prior to Java 7, Interned Strings or String Literals were stored as part of PermGen Space. In the above article, the way to configure the non-expandable or fixed PermGen has been provided, along with the diagram of where exactly it is located in memory. The PermGen space is always fixed. One more point to note here is that String Interns are candidate for garbage collection just like other Objects in Java.The default string pool size is 1009 and can be set through the switch -XX:StringTableSize

Java 7 Changes (String Handling)
So, With Java 7 - Oracle Engineers decided to move String Interns to the Heap. As mentioned above, this decision has to do with the following factors:

1. Fixed/Non-Expandable Size of PermGen (Leading to Out of Memory : PermGen Space)
2. String Interns are Garbage Collected like other Objects (Based on Reference Count)  
3. Set a Higher Number of Strings - Even upto Millions of Strings maybe Possible

Java 8 Changes (String Handling)
The above Java 7 Changes of String Handling continued into Java 8. Further In Java 8, the following change was made to the memory handling related to String Interning.

1. The default number of Strings in the String Pool is now 60013 as compared to 1009

At any point in time, you can check the String Interning and Pooling usage statistics in the JVM by using -XX:+PrintStringTableStatistics (Symbol Table Statistics)

Java 8 Changes (PermGen to MetaSpace)
The most impactful change to Java/JEE has been the movement from PermGen to MetaSpace. The following are the changes in Java 8:

01. Eliminate or Reduce Out of Memory : PermGen Space  Error. (PermGen is Gone) [PermGen Elimination]
02. Memory for Class Metadata is allocated only from the Native Memory. Earlier it was from Contiguous Java Heap Memory. [MetaSpace Allocation]
03. Virtually Unlimited (or Very High) Memory for Class Metadata from Native Memory. [MetaSpace Allocation]
04. Limit the Maximum Limit for Metadata in Native Memory Using -XX:MaxMetaspaceSize [MetaSpace Configuration]
05. Garbage Collection is Triggered once the MaxMetaspaceSize is reached [MetaSpace Garbage Collection]
06. PermGen and MetaSpace are not a Direct Movement (Some Items from PermGen have been moved to MetaSpace - There are some miscellaneous items moved to the Heap Space, like Interned Strings[PermGen <> MetaSpace, Heap Space Impact]
07. The earlier PermGen flags are ignored, like -XX:PermSize and -XX:MaxPermSize. If used, a warning is issued saying that these flags are no longer valid in Java 8.  [PermGen Compiler Flags]
08. Once the MaxMetaSpaceSize is reached, Out of Memory : MetaSpace Error is thrown. [MetaSpace Error] 
09. The default MetaSpaceSize on a 32-bit and 64-bit machine is 12MB and 16MB respectively. In contrast, the default MaxPermSize was always set to 64MB. [MetaSpace Default Size]
10. The Native Memory Size can be set by -XX:MaxDirectMemorySize (The default is 128MB) [Native Memory or Direct Buffer Size]
11. MetaSpace usage  is available from the verbose GC log using -XX:+PrintGCDetails
[MetaSpace Statistics]
12. PermGen had a fixed size but MetaSpace can Auto-Tune and Auto-Increase depending on underlying OS. [MetaSpace Size Efficiency]
13. Classes can be de-allocated concurrently without GC pause. It is more efficient than in PermGen as it required frequent GC pauses. [MetaSpace Performance]

Note: Many authors have erroneously stated that PermGen is part of the Java Heap. You have to note this fact that PermGen was/is never Java Heap - It was only allocated as part of Contiguous Memory as the Java HeapIn case of Metaspace, such a Contiguous Allocation does not exist, Metaspace lies in or is allocated in the Native Memory of the Operating System.