Know Your Stack - Scala to Java to Bytecode to Assembly

Many times in my career so far I could have made better decisions, solved problems much faster or even prevented problems if I had a deeper understanding
of the technology stack I was using.

To begin getting a deeper understanding of the Scala stack, we will trace a simple program through it's lifetime of becoming Java, then JVM bytecode and
then the final step before it's fed to the CPU monsters - assembly.

Starting with a high-level
overview like this is a good foundation on which to explore each step in more detail. Because I'm nice I'll give you lots of links at the bottom of this post so you can do just that.

All code relevant to this post is up
on my github.

Scala

Our initial block of code is a scala object with a main method. When the application is started this main method will be run and, as you can see below, it's going to print something stupid to the console and finish executing.

package kns

object KnowYourStack {

def main(args: Array[String]) {
val message = "Fat stacks, yo!"
println(message)
}
}

[/code]

Compiled to Java

When Scala code is compiled with sbt it's already output as bytecode. However,
we can use one of the java decompilers to show us what the java code would have looked like.

First I'll run sbt to compile the Scala. This turns our KnowYourStack.scala into two .class files as shown below:





I'll now use jd-gui to disassemble them back into java. Here are the results:

package kns;

import scala.Predef.;

public final class KnowYourStack$
{
public static final  MODULE$;

static
{
new ();
}

public void main(String[] args)
{
String message = "Fat stacks, yo!";
Predef..MODULE$.println(message);
}

private KnowYourStack$()
{
MODULE$ = this;
}
}

package kns;

import scala.reflect.ScalaSignature;

@ScalaSignature(bytes="\006\001\025:Q!\001\002\t\002\025\tQb\0238pof{WO]*uC\016\\'\"A\002\002\007-t7o\001\001\021\005\0319Q\"\001\002\007\013!\021\001\022A\005\003\033-swn^-pkJ\034F/Y2l'\t9!\002\005\002\f\0355\tABC\001\016\003\025\0318-\0317b\023\tyAB\001\004B]f\024VM\032\005\006#\035!\tAE\001\007y%t\027\016\036 \025\003\025AQ\001F\004\005\002U\tA!\\1j]R\021a#\007\t\003\027]I!\001\007\007\003\tUs\027\016\036\005\0065M\001\raG\001\005CJ<7\017E\002\f9yI!!\b\007\003\013\005\023(/Y=\021\005}\021cBA\006!\023\t\tC\"\001\004Qe\026$WMZ\005\003G\021\022aa\025;sS:<'BA\021\r\001")
public final class KnowYourStack
{
public static void main(String[] paramArrayOfString)
{
KnowYourStack..MODULE$.main(paramArrayOfString);
}
}

It looks like Scala creates both an instance and static class as a way to enforce the singleton pattern, which objects are. This is also the entry point
to the application, which is maybe what the Scala signature is doing here?

Next Stop: Bytecode

A JVM takes bytecode and JITs it
to machine code at runtime. It will even compile hot code that is being JITd frequently. The bytecode of our two generated .class files is below using javap.

➜ javap -c -p KnowYourStack\$

Compiled from "KnowYourStack.scala"

public final class kns.KnowYourStack$ {
public static final kns.KnowYourStack$ MODULE$;

public static {};
Code:
0: new           #2                  // class kns/KnowYourStack$
3: invokespecial #12                 // Method "":()V
6: return

public void main(java.lang.String[]);
Code:
0: ldc           #16                 // String Fat stacks, yo!
2: astore_2
3: getstatic     #21                 // Field scala/Predef$.MODULE$:Lscala/Predef$;
6: aload_2
7: invokevirtual #25                 // Method scala/Predef$.println:(Ljava/lang/Object;)V
10: return

private kns.KnowYourStack$();
Code:
0: aload_0
1: invokespecial #31                 // Method java/lang/Object."":()V
4: aload_0
5: putstatic     #33                 // Field MODULE$:Lkns/KnowYourStack$;
8: return
}

➜ javap -c -p KnowYourStack

Compiled from "KnowYourStack.scala"

public final class kns.KnowYourStack {
public static void main(java.lang.String[]);
Code:
0: getstatic     #16                 // Field kns/KnowYourStack$.MODULE$:Lkns/KnowYourStack$;
3: aload_0
4: invokevirtual #18                 // Method kns/KnowYourStack$.main:([Ljava/lang/String;)V
7: return
}

Interesting.... where did our string go? We can see it being loaded as constant 16 (ldc #16) and accessed later (getstatic #16) but where is the declaration? A lot of these instructions make a lot of sense when you've read/watched the links I've posted at the
bottom of this post.

Final Destination: Assembly

I've got 2 ravenous intel CPUs inside in my i5 Ivy Bridge; both of whom have an insatiable appetite for x86_64 soup. Let's see what we can knock up from our base of bytecode special ingredients.

To get assembly out of the JVM you need to pass two flags. We do that like this in sbt:

name := "know your stack"

scalaVersion := "2.10.2"

fork := true

javaOptions += "-XX:+UnlockDiagnosticVMOptions"

javaOptions += "-XX:+PrintAssembly"

Note also the fork settings is set to true. This is because sbt needs to fork a JVM with the settings applied. You'll also need to listen to this
smart guy.

We're now ready to cook.

After running sbt run my terminal erupts with lines of text that are hard to make out. Luckily I pipe it into a file, which on later inspection happens to be full of intel instructions. Here's a couple of snippets:

Start of the application

Running kns.KnowYourStack
OpenJDK 64-Bit Server VM warning: PrintAssembly is enabled; turning on DebugNonSafepoints to gain additional output
Loaded disassembler from /usr/lib/jvm/java-7-openjdk/jre/lib/amd64/hsdis-amd64.so
Decoding compiled method 0x00007f3419060250:
Code:
[Disassembling for mach='i386:x86-64'] [0m
[Entry Point]
[Constants]
[# {method} 'readLine' '([BII)I' in 'java/util/jar/Manifest$FastInputStream'
# this:     rsi:rsi   = 'java/util/jar/Manifest$FastInputStream'
# parm0:    rdx:rdx   = '[B'
# parm1:    rcx       = int
# parm2:    r8        = int
#           [sp+0x60]  (sp of caller)
0x00007f34190603a0: mov    0x8(%rsi),%r10d
0x00007f34190603a4: shl    $0x3,%r10
0x00007f34190603a8: cmp    %r10,%rax
0x00007f34190603ab: jne    0x00007f3419037960  ;   {runtime_call}
0x00007f34190603b1: xchg   %ax,%ax
0x00007f34190603b4: nopl   0x0(%rax,%rax,1)
0x00007f34190603bc: xchg   %ax,%ax
[Verified Entry Point]
0x00007f34190603c0: mov    %eax,-0x14000(%rsp)
0x00007f34190603c7: push   %rbp
0x00007f34190603c8: sub    $0x50,%rsp         ;*synchronization entry

Method Decompilation with Annotations

[Decoding compiled method 0x00007f3419062f50:
[Code:
[Entry Point]
[Verified Entry Point]
[Constants]
# {method} 'indexOf' '([CII[CIII)I' in 'java/lang/String' [
# parm0:    rsi:rsi   = '[C' [
# parm1:    rdx       = int [
# parm2:    rcx       = int [
# parm3:    r8:r8     = '[C' [
# parm4:    r9        = int [
# parm5:    rdi       = int [
# parm6:    [sp+0x50]   = int  (sp of caller) [
0x00007f34190630a0: mov    %eax,-0x14000(%rsp) [
0x00007f34190630a7: push   %rbp [
0x00007f34190630a8: sub    $0x40,%rsp         ;*synchronization entry [
; - java.lang.String::indexOf@-1 (line 1718) [
0x00007f34190630ac: mov    %rsi,0x18(%rsp) [
0x00007f34190630b1: mov    %edx,0x10(%rsp) [
0x00007f34190630b5: mov    %r9d,(%rsp) [
0x00007f34190630b9: mov    %ecx,0x8(%rsp) [
0x00007f34190630bd: mov    0x50(%rsp),%ebp [
0x00007f34190630c1: cmp    %ecx,%ebp [
0x00007f34190630c3: jge    0x00007f3419063481  ;*if_icmplt [

You can find the full output on
 my github. Just don't ask me to explain what all of those registers are used for.... yet.