Fix My Bugs

When I installed the latest Sun JDK 6 recently, one of the installer wizard screens had a JavaFX logo on it, and I wondered what it was, so I checked it out.  It turns out that JavaFX is a family of products from Sun to compliment Java, specifically geared towards designers and web developers creating dynamic web content.  The Java language was originally supposed to do this anyway, but it never really got off the ground for web content in the face of Flash and Javascript.  Java is a fairly traditional programming language, with it’s syntax based on C and is likely to give any web developers and designers, who aren’t necessarily classically trained programmers, a bit of a shock.  JavaFX contains a completely new language, which is declarative in style, and hence should make it easier to code graphical UIs and content based around concepts like screens, images, animations etc. rather than worry it’s users with threads, memory management and data structures.

Something that also caught my eye was it’s mobile cousin, JavaFX Mobile.  Sun has been successful in getting Java technology into mobile phones, a lot of mobile phones to be precise!  A few minutes of Googling didn’t come up with any actual figures, however I would go so far as to say that Java ME is probably the most ubiquitous general purpose computing application platform on the planet, in terms of numbers of in use handsets today.  I expect actual numbers are available to industry insiders who pay subscriptions to data gathering companies, rather than casual internet searchers!

According to Bruce Hopkins in an article on Sun’s web site, there are Three Reasons Why Your Next Java ME Mobile Application Should Include JavaFX Mobil. I’m not going to be using JavaFX in my next mobile application however, and I’ll explain why in relation to Bruce’s reasons one at a time.

Reason #1: JavaFX is a Brand New Platform and Language With a New Declarative Syntax

A declarative syntax is certainly useful when getting non-programmers to actually produce content for your application.  Looking at it from a mobile games point of view, I work with game and level designers who take an engine that I create and use it to build a game.  This is a two way parallel process, not something that happens in series.  A designer will create an initial specification including a feature set, the programmer will go away and implement that, and then as the level designer (often the same person as before) uses the engine to create and add game content (eg. levels), there will be a back and forth dialog with the programmer as new features are proposed and potentially implemented, bugs ironed out and the newly designer created content fed back into the game build process.

The tools that the programmer creates to give to the designer often contain a scripting language. It would be nice to build a fully featured graphical design tool specific to every game, but time constraints usually prevent that. The designer almost always has to get their hands dirty with some game scripting. This might be in a custom language written by the programmer, or it could be in a general purpose scripting language like Ruby, Lua or Python to name three. Either way, it’s likely to be heavily declarative in nature. The designer will use the script to declare game elements, like the level layout, player start position, enemy positions, enemy types and actions, scripted events keyed to triggers and timers, etc.  In this respect, JavaFX Mobile looks like it might be are great tool, a declarative language already built on top of the game implementation language (Java ME)…

However, as a programmer, have a look at this syntax and see if you’d give it to a designer?

package notifier;

import javafx.scene.image.Image;
import javafx.scene.image.ImageView;
import javafx.stage.Stage;
import javafx.scene.Scene;
import javafx.scene.text.Text;
import javafx.scene.text.Font;

var img = Image {
            url: "{__DIR__}email.png"
          }

Stage {
    width: 200
    height: 250
    title: "Mail Notifier"
    scene: Scene {
        content: [

            ImageView {
                image: img
                x: 0
                y: 0
            }

            Text {
                font: Font {
                    size: 16
                }
                x: 10
                y: 140
                content: "New Email Received!"
            }
        ]
    }
}

There are way to many curly brackets for a start, giving you all that nesting to contend with. And look at those square braces in the middle there, what the hell are they suppose to mean? There’s no way I would save any time by giving this to a designer, I’d be constantly hand holding them through the compile errors and reminding them when they need one type of bracket or another.  There are lots of existing scripting languages that have a much cleaner and more natural syntax for ease of use by non-programmers.  Creating your own language can also be an enticing prospect, but it’s not something to get into lightly. Programming language processors and compiler techniques are a highly complex area to deal in, but experience in it can be very valuable. In the tradition of Blue Peter, here’s one I made earlier:

# Level script file.
# Anything after a hash on a line is a comment.
# Case is not important.

# Initialisation section
code=start

# Dimensions (in tiles) and tile size (in pixels)
level dim=54,50 tilesize=16 id=mission3 number=3
      flags=!revealed, !complete, !failed, !shop
      nametext=Text_mission3_name
      introtext=Text_mission3_prologue
      outrotext=Text_mission3_epilogue
      respawn_id=respawn1

# Garret's start position
character type=garrett id=garrett posn=51,48.95 anim=idle:0:left

# 2nd guard in left tower
item type=map id=map3 posn=0,0 flags=hidden,infront
character type=guard id=guard2
          posn=10,23.95
          anim=walk:0:right
          pocket_id=map3
          action=patrolling_left
          end0=left:8,27 end1=right:8,19 look=10 stand=5 health=8 hit=2

# End of initialisation section
endcode

# Triggers that cause things to happen
code=triggers

# Map trigger
trigger action_id=reveal_map when
  test_flags item id=map3 flags=owned
endtrigger

# End of triggers
endcode 

# Actions that are run by triggers
code=actions

# reveal the map
action id=reveal_map
setflags level id=mission3 flags=revealed
endaction

# End of actions
endcode

The comparisons here should be pretty obvious. No braces or nesting to worry about, just 3 separate sections that contain code for different purposes. No import statements, which comes from the domain specific nature of the language. I myself can never remember which package or library a class comes from most of the time, someone scripting for me shouldn’t have to. The one common element between both examples is named parameters. That’s something I would consider essential in any language intended for a non-programmer to use, at least JavaFX has that right.

Reason #2: The JavaFX Platform is a Software Layer Above the JVM

On the surface, this is a pretty good reason to use JavaFX, if it wasn’t for the fact that all those millions of mobile devices available in the shops today won’t support it!  Only the very latest handsets will have support for it included, and the numbers of those sold are going to pale in comparison to the existing device take up figures.  That’s the big problem with mobile development, the massive device fragmentation problem.  I expect that it will take at least 1 years for any new Java ME feature to make it into handsets, and then at least another year on top of that before enough handsets are available to make it worthwhile developers using that feature.  I’m assuming that JavaFX mobile needs device manufacturers to make devices that support it.  It may be possible to code using the JavaFX script and have the source compile to standard Java bytecode that runs on existing MIDP 2.0 devices.  If that’s the case, then fantastic, as long as the required supporting libraries aren’t 100′s of kilobytes big!  There are still phones that only support jars of 64k or less, I’m looking at you Nokia Series 40 v1 devices.

Reason #3: Data Binding with the Bind Keyword

I’ll concede that this feature is quite nice, and certainly makes trivial examples like updating a text field when a slider moves nice and easy.  I’d like to see some more heavyweight usage examples though, and I’d like to see what non-programmer designers make of it.  If programmers are going to be writing the scripts, they’ll probably quite quickly run into it’s limitations and end up writing custom code for all sorts of circumstances.  I won’t rag on this feature too much at the moment though since I’ve got more to learn about it.  One thing I will say though is that it’s simply one language feature, and not something I think should weigh heavily on any decision to use JavaFX in a project or not.

So, there’s a brief look at JavaFX Mobile and why my next mobile application won’t use it. It’s certainly interesting, and might have more relevance to non-game applications, but I predict it won’t be a feature anyone shouts about loudly for very long!

This is a copy of the feature I wrote for GamesOnDeck.com, the mobile portal of Gamasutra. It’s a good article to start my blog off with, and hopefully fairly typical of the output I want to produce in the future. I’ve not changed it from the original, although I think the general layout and formatting of this WordPress post is much better for a technical article than the rather cramped GamesOnDeck layout. I’m particularly pleased with the Google Syntax Highlighter WordPress plugin and how it shows source code.

Finally, you can download the source code referenced in the article from here.

Introduction

There are few things worse for a programmer than the delay between altering some lines of code and seeing those changes working in game. Developing J2ME games is especially prone to build process delay, with many steps required between compilation and final jar file output. Add to this the complication of many different device emulators with their slow start up times, poor keyboard response and limited debugging support. Mobile Java programming can be speed up and even made a pleasurable experience with the help of some J2SE code which simulates the J2ME APIs. This article will lead programmers through the creation of a deceptively simple J2ME simulator which speeds up programming, improves debugging, facilitates rapid prototyping, aids the creation of device independent code and frees you from build processes that may take minutes to complete.

What’s wrong with emulators?

I’m not going to list point by point all the things I personally dislike about using device manufacturer’s emulators for J2ME development. I’m sure if you’ve spent any time developing J2ME applications you’ve felt there must be something better. Firstly, there are just so many SDKs and emulators available. Installing, maintaining, learning and integrating them into your development process is a time consuming and cumbersome task. Secondly, actually using them frustrates at every stage of programming. They are slow to invoke, offer little or no feedback when errors occur, and debugging support is usually unreliable or impossible.

^{Figure 1.What you usually do.}

If you are an experienced J2ME developer who knows how to write memory efficient, jar size and performance conscious code, I would suggest that your primary need for the majority of the programming work on a project is simply a J2ME environment that supports arbitrary screen sizes and all the relevant APIs your code utilises. You can get this in a tool that can be invoked in the blink of an eye both for normal execution and source level debugging, much more simply than you might think.

^{Figure 2.What you could do.}

Simulating J2ME with Desktop Java

Implementing the necessary J2ME APIs using pure Java turns out to be quite simple. The classes in CLDC 1.0 and 1.1 (java.io, java.lang and java.util) are all derived from their J2SE counterparts, so literally no wrapping or re-writing is required for the base standard library. The javax.microedition.midlet package is simple, well defined and easy to implement. The javax.microedition.lcdui package contains classes that (for the main part) have AWT counterparts, and those that don’t, you probably never use anyway, so there’s no need to implement them! You can probably do without the rest too, although javax.microedition.rms comes in handy, it’s just file storage and again, easy to implement.

^{Figure 3.Simulator Class Overview}

Figure 3 shows all but one class (MIDletStateChangeException) that needs to be implemented to create a functioning J2ME environment. The boxes in red are the main framework of the application and provide the entry point, window to display the midlet upon and keyboard input. The boxes in grey are the J2ME API classes that midlets use. These hook into each other and the simulator classes. Image, Graphics and Font are implemented using their AWT equivalent classes, shown in blue.

Start Coding

The MIDlet class contained within the javax.microedition.midlet package is the starting point of all midlet development and is the obvious place to look first when creating a MIDP execution environment. All midlets have to provide a class that extends MIDlet, and the 3 abstract methods that need to be overridden are the almost completely self explanatory startApp, pauseApp and destroyApp.

To get off the ground, our MIDP simulator needs an implementation of the MIDlet class with some public methods that call the abstract midlet lifecycle methods. Since these refer to the trivial MIDletStateChangeException, we should add this too.

Then we’ll need a ‘main’ application class for command line invocation that loads the concrete midlet class and invokes the public method that calls startApp. The following code shows what’s needed to load and invoke a midlet.

// Create our AWT main frame
frame = new FrameAWT (midlet_class_name);

// Load and instantiate the midlet
midlet = (MIDlet) Class.forName(midlet_class_name).newInstance();

// Call the midlet startApp method
midlet.amsStartApp ();

The main class should probably be in a package of its own, with a suitable name. I’ve chosen Jammy as the name of my J2ME simulator. I prefer single words over compound words or complex names that aren’t very catchy. With just these 3 classes in place we are able to compile a midlet and execute it by invoking it with Jammy. If you have simple midlet built into a jar, you can invoke the simulator using a command line like that shown below.

java –classpath bin;BasicMidlet_v1.0.0.jar com.longsteve.jammy.Main BasicMIDlet

Adding a display

Just running a basic midlet is ok, but we need to add a display. Since we’re writing our simulator in Java, we can use one of the windowing toolkits available with J2SE. We could choose Swing and use JFrame and its companion classes, but I’ve often had trouble with threads when using Swing. As far as our midlets are concerned too, all we need is a window and a canvas, we don’t need all the fancy widgets of Swing, so I’m going to keep it simple and use the AWT.

In J2ME, midlets use classes in the javax.microedition.lcdui package to interface with a display. The Display class provides the static setCurrent method, which a midlet uses to set a Displayable object visible on the screen. Canvas is the primary Displayable subclass a game will use. The midlet is coded with its own subclass of Canvas, which draws the game frames in its paint method, using a Graphics context.

We’re going to need to create the javax.microedition.lcdui package and add Displayable, Canvas and Graphics to the simulator. This will mean adding AWT Frame and Canvas classes, and interfacing these with the MIDP classes. Later we’ll add the MIDP Image and Font classes and expand the basic midlet even further into something resembling a game.

Main window

FrameAWT extends the AWT Frame class and provides the main window for the midlet display. In order to know how big the window should be, there’s a command line argument to the main class that specifies the dimensions for the midlet canvas size. There’s nothing to stop these dimensions from being cleverly worked out by referring to known device details, or by inferring the size from some metadata left over from a build process. Command line arguments are convenient for now though.

The canvas

CanvasAWT extends the AWT Canvas class and handles the screen drawing and keyboard/mouse input. CanvasAWT is linked to the current midlet Displayable object by containing a reference to it. A public method on Displayable called invokePaint() is used to actually invoke the midlets Canvas.paint() method. CanvasAWT maintains a MIDP Image object, which is used as the midlet screen. Each time the AWT paint method in CanvasAWT is called, it calls the midlet Canvas paint() method with a MIDP Graphics object derived from the screen Image. When the MIDP Canvas paint() method returns, the screen image is painted to the AWT Graphics context passed to the CanvasAWT paint() method.

The code below summarises the the CanvasAWT.paint() method. The invokePaint() method is part of our implementation of the MIDP Displayable class. When a midlet calls Display.setCurrent(), our Display class actually sets the ‘current’ object in CanvasAWT, so all windows paint requests get forwarded to the midlet.

public javax.microedition.lcdui.Displayable  current;
public javax.microedition.lcdui.Image        midp_screen;

public void paint (java.awt.Graphics g)
{
   javax.microedition.lcdui.Graphics midp_graphics =
                                     midp_screen.getGraphics ();

   // Call the midlet paint method with a J2ME graphics context
   current.invokePaint (midp_graphics);

   // Draw the midlet screen image to this canvas, scaled up
   g.setClip(0,0,awt_canvas_width,awt_canvas_height);
   g.drawImage(midp_screen._image,
               0,0,getWidth(),getHeight(),
               0,0,midp_screen.getWidth(),midp_screen.getHeight(),
               this);
}

The AWT drawImage() method also handily scales up the MIDP screen image to the required size, so you can resize the main window and see a zoomed in view of the midlet. I hate squinting at tiny emulator windows on my desktop, even the 2x zoom offered by some device emulators isn’t enough. By default, Jammy creates its canvas 3x the required size, but a command line parameter could be used to set this as you like.

Keyboard input

Keyboard input is handled by CanvasAWT in combination with the Displayable class. CanvasAWT implements the AWT KeyListener interface, and handles keyboard events, passing the J2SE key code straight to the invokeKeyPressed() method of the current Displayable.

Our Displayable class contains a table of J2SE key codes and their mappings to MIDP key codes and game actions. It would have been easy to simply re-use the J2SE key codes here, instead of inventing (or borrowing) J2ME codes. However, the J2ME Canvas class definition actually specifies the constant values for the codes (e.g. KEY_NUM0 is 48), so using the same ones will make the simulator directly compatible with existing midlets in this respect.

Image and Graphics

MIDP Image objects are implemented fairly simply by using a java.awt.BufferedImage. BufferedImage objects can be created with a width and height, like MIDP Image objects, or created from input streams using one of a number of image support methods or libraries. The Java 5 ImageIO class contains support for PNG images, as does the Advanced Imaging (javax.media.jai) package. I’ve found these methods of reading PNG data unreliable though. By far the fastest and most robust PNG library for Java (that I’ve used) is the library from sixlegs.com. The library jar is under 50k, licensed with a library exception to the GPL and succeeds in reading optimised and crushed PNGs where the built in Java support fails.

MIDP Graphics objects are created directly from Image objects and are implemented using a reference to a java.awt.Graphics object, created from the java.awt.Image. Most of the J2ME functionality is simply forwarding on method calls to the underlying AWT Graphics object, e.g. drawLine(). There are some differences though, which include the image anchors commonly used with J2ME for the drawImage() and drawString() methods, see the code below for an example. Things do get more complex if you start implementing any enhanced MIDP 2.0 methods like drawRegion(), which includes image transformations like rotation and mirroring. These are by no means impossible though, and some investigation into the AWT Graphics2D and AffineTransform classes should provide all the methods required.

public void drawImage(Image img, int x, int y, int anchor)
{
   // default anchor
   if (anchor == 0)
   {
      anchor = TOP | LEFT;
   }   

   // Work out the x and y offsets given specific anchors
   switch (anchor & (TOP|BOTTOM|BASELINE|VCENTER))
   {
      case BASELINE:
      case BOTTOM:
         y -= img.getHeight();
         break;
      case VCENTER:
         y -= img.getHeight() >> 1;
         break;
      case TOP:
      default:
         break;
   }

   switch (anchor & (LEFT|RIGHT|HCENTER))
   {
      case RIGHT:
         x -= img.getWidth();
         break;
      case HCENTER:
         x -= img.getWidth() >> 1;
         break;
      case LEFT:
      default:
         break;
   }

   // Draw the AWT image within the MIDP image to our AWT
   // graphics context
   _graphics.drawImage(img._image,x,y,null);
}

Font

Closely related to the Graphics class is the MIDP Font class, and any implementation of Graphics cannot be completed without it. By using java.awt.Font and java.awt.FontMetrics objects, all of the functionality is fairly simple to code. Some elements aren’t quite perfect, like underlined text support, but if your game uses a bitmap font, you’re not going to worry too about any text limitations.

Extras

Implementing just eight J2ME classes and three application classes is enough to start developing games using your simulator. You have all the components you need, and you may even find that you can run some existing games you already have. However, if you try running anything complex, you’re likely to run into class and method not found exceptions. What’s great though, is you can immediately see any components you need to add in order to flesh out the simulator.

Assuming you try a MIDP 1.0 compliant game you’ll almost certainly find it needs an implementation of the javax.microedition.rms record store package. Stub this out with some dummy code and you’ll be surprised at how much will now run. Adding actual file storage isn’t hard either using Java’s excellent file and stream IO classes.

Following that, it will probably be the additional classes in the javax.microedition.lcdui package that need adding. Command, and CommandListener should be the first, allowing access to device ‘softkeys’. You’ll probably not need any of the forms classes, since most games don’t use these. Figure 4 below shows Jammy running our old X-Change game.

^{Figure 4.Jammy running X-Change}

Following on from the basic MIDP 1.0 functionality, MIDP 2.0 additions slot in fairly obviously, I already mentioned the graphics transformations. If you really want to get a wide range of existing games working, then adding the Nokia UI would be good too. If you do MIDP 2.0 first, then the Nokia UI can be implemented completely on top of it.

Audio

The javax.microedition.media package can be added for audio support. You can use the javax.sound.midi and javax.sound.sampled J2SE packages if you’re using Java 5 or above. Again, stub out the classes first so that any midlet you are attempting to run actually works without crashing, then you can implement the functions and get real sound playing.

Bluetooth

Developing midlets with Bluetooth can be awkward since none of the manufactures J2ME emulators support actual wireless transmission. They all simulate it between running instances of emulators. Any actual integration with a real Bluetooth device is going to need testing on a phone, which slows the development and debug cycle immensely. There are javax.bluetooth (JSR-82) packages for J2SE available today. One GNU LGPL implementation is called BlueCove and can be dropped into your classpath for development. Another JSR-82 implementation is called avetana and is available for free trial, with a very reasonable fee for continued use.

3D

Once you’ve experienced the benefits of developing with a J2SE simulator, you’ll want it to support all the common APIs your games utilize. The Mobile 3D Graphics API (JSR-184) is increasingly important to J2ME games. Hybrid Graphics have a development product called Rasteroid which is a J2SE implementation of JSR-184. Adding this to your simulator won’t quite provide binary compatibility with midlets due to it’s direct dependency on AWT classes, but it will allow you to develop 3D midlets with only a couple of minor source code tweaks.

The Benefits

You might wonder if programming your own J2ME simulator is worth the effort, in the face of existing emulators from device manufacturers. My own simulator application grew out of a level design tool for a J2ME game. I wanted to use my game engine within the Java based design tool to provide rapid feedback to the designer. The simplest way to do this was to include the necessary J2ME support classes and methods within my design tool so the game engine class could be dropped in without any source code changes. While coding the design tool, the speed of standard Java development (compile and run) along with instant source code debugging made me wish I could carry on doing it for the rest of the game code.
In the end, I re-factored the J2ME support code into a stand alone application that ran my complete midlet. Then, over new projects, I’ve gradually implementing more supporting J2ME code as it was needed.

When I first thought about my design tool requirements, had I known of Mpowerplayer I probably would have used it, rather than write my own tool. Mpowerplayer is a great development tool and the company have built a quality service offering around it. There are additional benefits to owning your own in house tool however, many of which weren’t immediately obvious.

Really Rapid Development

An ant build process, with its compile, obfuscate, preverify and jar steps will take several seconds. You can set up a project in your IDE that includes your simulator classes, game classes and resource path, hit the ‘Run’ button and see you code running immediately. If you use Eclipse you won’t even need to compile first, it’s like developing any desktop Java application. If your standard studio project environment includes any sort of meta data about your current device target, you can build this into the simulator and have it start automatically at the correct screen dimensions. No need to switch emulators to perform the bulk of the porting work, retargeting a game at a different screen size.

Debugging

With a J2SE simulator you’re essentially coding your midlets in a pure Java development environment. As well as hitting the ‘Run’ button in your IDE (did I mention that already?), you can hit the ‘Debug’ button, and step through the source code. How long have you been developing J2ME games without source code debugging? This feature alone is probably the single best reason to ditch traditional emulators. I know they are supposed to support debugging, and some of the modern ones don’t do a bad job, but source level Java debugging in your IDE and debugger of choice is much more mature. You can also take advantage of numerous Java profiler tools.

Game Design/Prototyping

Having nailed the rapid development cycle, you’re going to be able to knock up prototypes faster. These builds can be given to designers, testers and anyone else to play with on their PCs, and all they need is a Java Virtual Machine installed. No need for numerous emulators, all to run different builds of the game.

How about a design tool that incorporates a build of your game using your simulator as the runtime? Designers can code levels, or artists can change tile sets and instantly see the results. You could add another AWT window to a special build with some sliders or checkboxes that allow game engine adjustments, so testers can tune difficulty parameters in real time. Artists can drop PNG images directly into the resource directory to see real time changes in the game, rather than fiddling with zip files and updating the JAD files.

Post Development

Having control of the environment your games run in brings further benefits after a game is complete. A lot of post development work is involved in obtaining submission material for operators, screenshots and video footage. Adding a screenshot facility is simply a matter of taking the MIDP screen buffer and writing it to a PNG file (inside CanvasAWT):

java.io.File f = new File (“screenshot.png”);

javax.imageio.ImageIO.write((BufferedImage)midp_screen._image,
                             "png", f);

Another potential time saver would be the ability to output video files of a game session. This would benefit QA too, with testers being able to record details of a bug. The Java Media Framework contains functionality to write AVI files, and could be hooked into the paint output of CanvasAWT, like the screenshot code.

Having a set of code not constrained to a J2ME execution environment opens up countless possibilities for running your games. You can demonstrate J2ME games to potential clients using a laptop and a projector, rather than hand them a phone to squint at they get a full screen presentation. You can plan for future developments like VGA screen size phones simply by setting your simulator display to 640×480. Running your games as applets on the web is another obvious possibility, as are Pocket PC and PDA type devices. Java native compilers might also offer a quick route to alternative executable versions of your games for non-Java platforms.

Conclusion

I hope this article has shown how simple it is to write your own J2ME simulator using pure J2SE Java. By becoming (mostly) free of manufacturers device emulators, your full game development cycle can be positively enhanced. Stable source level debugging alone should have J2ME programmers running! Taking control of your midlet execution environment might also open up surprising new avenues for your games.