Wednesday, July 2, 2008

Plan for Robocode 1.6.2

We just released Robocode 1.6.1 Beta for community testing. While we are waiting for at least one email with feedback (we really miss it), I will present detailed list of tasks for next version 1.6.2.

RobotPeer redesign - messages instead of shared state

Each robot has it's peer which takes care about the real truth about that robot. Things like energy, velocity, direction and also intentions (settings or commands) are stored there. Battle is reading them each turn and modifying according to rules. Robot is accessing that location using RobotAPI. This location is represented with class RobotPeer.
There are two problems currently. First is that both threads battle and robot's are accessing RobotPeer, so data there need to be synchronized. Current implementation does that per call. So subsequent calls could return different values. And as it is, we couldn't do any better, because bigger granularity could lead to deadlocks or at least to slowdown.
Second problem is that there is fixed cost per call for such synchronization from both sides, robot's and battle's.

What we plan is that we split RobotPeer class to two separate parts. First will serve to robot and second will serve to battle. They will be synchronized once per turn. Let's call first part RobotProxy and RobotPeer the second part from now on.
Great moment for such synchronization is when robot calls execute() method. As everyone knows, robot is blocked till next turn there. We could gather all settings and commands set by robot from last turn and make message from them. Then we could send such message to battle's part.
During processing of turn facts about robot are changed, also new events are produced. Such data need to be sent back to robot during resuming after call to execute(). That data will be merged to information set hold by RobotProxy. RobotProxy will serve as local cache for such info.

We may also wish to decompose hosting of the robot from actual battle logic.

Battle recording and replay - listener and producer

  • New version of independent class inside of BattleManager class, next to Battle class. May be called Recorder ?
  • Still controlled thru BattleManager and sending BattleEvents like Battle does
  • Recording should be implemented as IBattleListener
  • Replay will just feed dispatcher of BattleManager
  • Think about IBattleControl interface and where to put method for saving and loading recorded battles
  • Should be possible to go to any turn, and also rewind the battle etc.
  • Should be able to records robot renderings as well as print outs
  • Possibility of saving/loading the Replay to a file, e.g. XML

Introduction of interfaces of future modules

As preparation for modularization in 1.6.3, we could do some of the steps earlier. The difference is that in 1.6.2 we will not move implementations in namespaces and we will not break compatibility. My guess is that full modularization will need such non-compatible changes. I'm not talking about RobotAPI, but ControlAPI. So we might even need to release it under 1.7 version instead of 1.6.3.
It will be big change. So some of the work ahead could help us manage it better.

Low priority tasks - nice to have

  • OptionManager - components subscribe for changes in setup/options/configuration
  • Reduce complexity of TeamPeer, ContestantPeer, ContestantStatistics, TeamStatistics
  • Decouple battle save/load from BattleManager
  • Adding the music from Hans HiScore
  • ATW battle events dispatcher, single point dispatcher on AWT thread
  • Robocode log to window - not only to console but as well to GUI window

Tuesday, June 10, 2008

Enterprise Architect - strip some GUIDs

Sometimes you need to use EA module as a template for multiple similar models. So you strip GUIDs and you are fine. Problem appears when you need to share some of elements in shared module and when you care to keep reference to that shared elements in new module from template. We use EA together with SVN for packages (xmi). In one project we have several of them. Whole solution is made of many projects.

We have library (package) of known UML elements (like components or systems), which are shared across all projects and are used/instantiated in packages of models. Quite often we need to use one of packages as a template for another one in same model. Specifically, my use-case is that template package is containing instances of components, which are defined in shared package. But there are also other components (in template module) which are templates and which need separate identity after used as template.

ea ea

Because EA doesn't support such use-case we were forced to redraw it over and over again. I really hate repeating what was already done. I'm also lazy to do that. So I implemented tool which does the work for me.

Inside of XMI entities have it's identity referenced using GUIDs. So we need to strip GUIDs in template module, but only some of them. Those which link to common package should remain linked. And those elements which are template need to be striped.

Simple explanation how tools works is that I parse out all GUIDs from template module. Then I parse GUIDs from shared modules and match them. If I have found GUID in shared module, I should keep it in new copy of templated module. Otherwise I replace it with newly generated GUID. Whole thing is just quick and dirty solution. I don't parse XMI properly and don't care about this piece of code too much.

Usage: Export your shared modules to XMI. Export your template module to XMI. Use tool.

StripSomeGUIDs.exe [shared-xmi-file-1] {[... shared1-xmi-file-N]} [xmi-file-to-strip] 

Import result from [xmi-file-to-strip].new following way. You shouldn't strip GUIDs again, because it was already done.

ea ea

Code is there.
Enjoy!

Sunday, April 13, 2008

Module management and Dependecy injection

I'm trying to choose some framework which will help us to manage modules and dependencies after modularization of Robocode.
There are several criteria
  • small
  • fast
  • easy to use
  • low impact on code
  • easy to configure when user adds new module at runtime

I would like to hear your opinion about weight of criteria above.
I have hard time to choose right one, because lot of Java specific names are new to me. Half of articles on the Internet is kind of holy war again, so it is thought to find informed and balanced article.
I will  try to summarize what I've learned about possibilities.

We have three problems
  • 1) how to friendly add .jar plugin to classpath
  • 2) how to manage dependencies
  • 3) how to manage different versions of plugins or their dependencies on same classpath
  • 4) we also need to consider licensing. I'm not an licensing expert, though.

Solution for 3) and also 1)

It is quite difficult problem.
There are some standard solutions to the problem
OSGI
http://felix.apache.org/site/index.html
http://www.eclipse.org/equinox/
http://www.knopflerfish.org/

JPF
http://jpf.sourceforge.net/

I propose to ignore the problem, because solution is too heavy for our needs. Ignoring c) may lead to necessity of recompilation of cascade of plugins when some of them will change heavily. I believe that we could live with that.

Update: I have found concierge, which looks pretty small = 80 KB. I should try to use it.

Solution for 2)

This problem is nicely solved by various Dependecy injection frameworks.

Spring, Spring JavaConfig
+ de facto standard
- too heavy

Guice
+ fast
+ quite nice
+ type safe
* no that big, but still ....
- no external configuration (XML)
- code will become dependent on it because annotations

PicoContainer
+ small
* all wiring is done in code
- no external configuration (XML)

NanoContainer (buid on top of PicoContainer)
+basic xml configuration
+ still very small (176KB)
+ able to solve problem 1)
- lack of documentation

Conclusion

For our use case I like most PicoContainer+NanoContainer.
It solves problem 1) and 2), with small size, with possibility of XML configuration, and our components could be independent on it's namespaces. I tested it on small test project with one core and one plugged jar, and with XML config. I was also able to pass parameter to constructor from XML.

I'm interested to hear other opinions. Thanks!

Friday, April 11, 2008

Robocode Modules - draft

I quickly put together proposal of modularization. I didn't yet checked it against current code, so I probably missed something. The names do not match current names of classes. Interfaces are not yet designed at all. It's just first draft. Robocode

Saturday, March 29, 2008

Robocode 1.6 explained

Why ?

During last Christmas I was seeking for some new toy and remembered Robocode, game or competition for Java programmers. I got an idea that I would rework Robocode for .NET. I sat down and wrote an email to Flemming N. Larsen, owner of Robocode. Soon we realized that we had common opinion about the need for support of other languages in Robocode. We were discussing various technical solutions how we could do it, including IPC messages and complete rewrite. Then I came with idea of IKVM as possible solution. During few days we hacked the prototype. Yep, Robocode for .NET exists, it is able to run both Java and .NET robots, paint the battlefield and it is pretty incomplete and unstable.

Update: 17.02.2010 Finally, Robocode .NET version works. It looks different than described here :-D

During prototyping I realized that I couldn't run AWT on top of IKVM, so we splitted Robocode UI from core and drafted new UI in WinForms.
This was first obstacle, need of split of UI from core "rules". I also hacked robot loader to be able to load .NET robots together with Java robots in same battle.

I started with reimplementation of robot base classes for .NET, because it support properties and naming convetions are different. I would like to make the robot classes look natively to .NET beginner programmers.
There I hit second obstacle, because pre 1.6 versions of Robocode could recognize only robot inherited from Robot class. So when you just inherit .NET robot from Java robot, there will be lot of methods from Java object or just named by Java convention. And this looks ugly in .NET.

Later when Flemming implemented drawing of the battle field (he learned some of GDI+ on it ;-), we hit third obstacle. WinForms are running in single thread only, but Robocode is running lot of threads and doesn't care about synchronization of UI much.

So the prototype showed us, that it is possible to share core of Robocode, but the code should be decomposed and refactored.

Version 1.6 is first step, introduction of robot interfaces, to get rid of necessity to inherit from standard Robocode base robots. I believe that the changes we do are not only for possibility of .NET version, but also for beauty of pure Java Robocode.

How Robocode works

Because lot of people don't have so much knowledge how Robocode internals works, let me first introduce you to the problem area. This is described from viewpoint of robot and oversimplified. Robocode
During start of battle round, for each "robot specification" RobotPeer instance is created. This class instance is containing "all the truth" about robot. It is then creating robot instance of "user" robot and start the "robot thread". This thread is that thing which is powering user code of robot, usually calling bunch of get() calls on base class and then it calls execution of some blocking command (for example move or fire). Blocking means that robot thread is blocked in the engine until engine cames to next tick. During tick all robots are examined (by battle thread) for commands issued, position, speed and energy and rules are applied. Then robot threads are resumed again, and redirected into EventManager to evaluate conditions of events. Events are called (still by same robot thread) back to user-land code of robot event handler. Usually there is another blocking command issued from that event handler, which is just nested (recursive call) on the stack.

Event interfaces

On the picture above you already can see one of new interfaces. The interface of event handler. Before 1.6 there were always the methods with same signature as on new event interfaces, they were empty handlers on robot base class and you could override them. This is still true. But now you can also decompose such event listener to some other class. You can see the example of it on AlienComposition robot from new sampleex package. I'm using such decomposition to hide the methods/handlers with Java-like naming convention for .NET base robots. Each robot kind (junior, standard, advanced, team) has some specific events, so there are multiple event interfaces. You can examine it on the picture below.

Robot interfaces

Till version 1.6, every robot should be inherited from Robot or JuniorRobot class to be recognized by Robocode engine. This is another degree of freedom which needs to be unlocked to implement nice base robots in .NET. So now you are not forced to inherit from Robot, instead you just need to implement IBasicRobot, IJuniorRobot, IInteractiveRobot, IAdvancedRobot or ITeamRobot. Each of them will allow you to provide different event handlers and you will be also given by IRobot*Peer which will support appropriate commands. There is also another improvement, so that IAdvancedRobot is not forced to be IInteractiveRobot. In other words, such advanced robots will run faster as they will not need to handle UI events.

I hope that one nice day someone will also use this interfaces to implement nice base robot for Jython or other languages on top of Java.

Peer interfaces

Till version 1.6, security of commands and queries executed by "user" robot code were guarded by robot base class. More advanced base class you inherited more possibilities you got. But more duties as well. Base classes were calling RobotPeer, which has full set of methods,which are not guarded any way. Now when we would like to allow you to just implement the interface, instead of inheriting from base class, we need to give you opportunity to issue commands and queries other way. But we still need to assure that you will not use more power than belongs to your robot kind. So we put protecting shield between base robot and RobotPeer. It is implemented by *RobotProxy classes, which implement I*RobotPeer interfaces. Now when you would like to implement issuing of command in your robot of base robot, you should call thru I*RobotPeer interfaces. This way we created secure layer and allowed robots to inherit from custom classes.

For most of users, inheritance from base classes is still right way how to do it. We are not trying to replace it, as it serve well. This is rather opportunity for very advanced coders to create new base classes with different architecture for others, or for other languages.

All together

All the new functionality described above should work transparently for all existing robots. The new features could be now used only when Robocode is started with -DEXPERIMENTAL=true command-line option, but this limitation will probably disappear in future versions.
I must admire Flemming for hard work on Javadoc for new interfaces. Also documentation of other classes was improved by him.
Robocode interfaces
I believe that decomposition we did is good thing, even for pure Java version, because separation of concerns.

In the pipeline

As you might guess, not all problems found during the work on the prototype were solved in Robocode 1.6. We are working on next steps.

First of them will be rework of RobotPeer, which will address synchronization issues caused by display thread. Most of this work in progress has been already done in this branch. RobotPeer was split into several parts by usage. By robot thread, by battle thread and by display thread. Data about robot state are also decomposed into separate classes, so they could be now fully synchronized. We also moved handling of interactive events from UI thread to robot thread by posting it thru EventManager queue, so you could change priority of such events. If you look at RobotPeer class now in 1.6 you will see big spaghetti implementation, this work should help it a lot. I didn't yet back-merged the changes to .NET branch, so I don't know yet whether it solved UI deadlocks. I will write another article when we get there.

After then we will need to split carefully UI from core, so that core could be reused in .NET. This was already drafted in .NET branch. I'm also thinking about more advanced modularization, plug-in infrastructure. Ideas are welcome.

When we will have that, we can build .NET version on top of it. We could implement UI visually compatible with Java version, or different, if someone will invent something better. The battlefield will be the same of course. We should also think about version control options. For now we are just merging the branches.
After then I dream about IronPyton and IronRuby robots built on top of it ;-)

Feedback

We need lot of feedback and we need it now when 1.6 is Beta. I guess that we will have few weeks of time, before we reach release and will merge interfaces branch to the trunk.

We need lot of testers. Please try to run your historic robots, they should run just OK. Please try to play with new interfaces, so that we could get your feedback. The interfaces will become new API between robot and engine. That means that they will live with us long time, so please review them and shout now, if you don't like them (and have better ideas). Please try also to broke security of the solution, in both legacy and experimental mode.
Questions are also welcome.

Thanks a lot!

Update: 02.04.2008
Just to make it clear: 1) Robocode will stay 100% Java, my IKVM extension will be just build on top of it, so you will be able to run on any Java enabled HW or OS as usually.
2) We still don't have plan how Java only users could run .NET robots, for example in Roborummble. Rednaxela is proposing JaCIL. Ideas and prototypes are welcome.
3) We are looking for someone who will help us design and prototype plugin, which will allow to load robots in other languages running on JVM.

Saturday, March 15, 2008

Monday, March 10, 2008

Smooth transition from C# to Java SE

Robocode

In free time I'm playing with Robocode. At the start my idea was to migrate it somehow to .NET and run away from Java as soon as possible. I use IKVM to wrap Robocode. I learned that Java GUI, which is AWT in this case, could not be translated to WinForms by current version of IKVM. Current Robocode is monolithic application, where UI is in same jar module as the core. I was looking for some IDE which will help me with Java and to introduce/refactor UI interfaces and split of UI from core. Currently I'm buried much deeper in Java part of Robocode, but that will be another story.

Update 17.2.2010: Robocode for .NET took different direction, but it's here.

IntelliJ IDEA

Because I'm mostly C# guy, I wanted to use some IDE which will behave similarly like VS. IntelliJ IDEA won the race, because I heavily use their ReSharper for C# in Visual Studio, luckily in the original IDEA key-binding mode. JetBrains were also so kind that donated open-source license to Robocode project. I needed to do only two things to feel at home in IDEA:
1) Install TabSwitch plug-in
2) In File/settings/keymap choose VS, make copy, in tree seek for TabSwitch plugin and set Ctrl-Tab key for SwitchBetweenTwoTabs.

IKVM

is compiler, which translates Java bytecode to .NET IL. In other words you take .jar and compile it with IKVMc to get .NET assembly. Java SDK is also compiled this way so your .jar is running on top of Java framework, but natively on .NET runtime. There is small IKVM runtime for Java framework, similar as runtimes for different platforms. You could event turn on debug flag and you will be able to debug Java code inside VS, pretty cool.

Friday, March 7, 2008

Oracle documentation sucks

Warning, rant below:

I'm trying to learn how to work with XML in PL/SQL. So, I need some examples and reference manual. For C# or WINAPI I'm used to search for method names on google and usually I get link to MSDN. There is detailed explanation about method, parameters, types, usual and special values. There is often example of usage.

That's not true for Oracle documentation. Look at this . Procedure doesn't have it's own page, name of link is cryptic, so google has small chance. The documentation is poor, missing hyper-links to types of procedure parameters. This is documentation given by Oracle programmers, not by documentation writer.

There are zilions of users of Oracle database struggling with basics, just because of bad documentation. If you are searching for solution to your question, you usually find some discussion group with bad example of usage from some other confused user and below some non-authoritative response from people repeating same answers twenty times a day.

Another problem is that there is no documentation standard for in code documentation of PL/SQL. There are some open-source tools which are trying to map for example javadoc tags to procedures, but nobody knows about that.

I hope I missed something and someone will tell me where are standards described and who is evangelizing about it. Someone will give me link to real reference documentation to Oracle built-in packages.

Friday, February 22, 2008

Lisp lectures IIa - Higher-order functions

lisp lectures
I'm done with lesson IIa. That one was pretty hard. In Lisp it would be probably quite easy, but not in C#.
First thing, you need to switch from delegates to expression trees. There is no major problem to wrap delegates from my last article using Expression<>. Only problem is funtion folding. Consider this:
Expression<Func<double, double, double>> average = (y, x) => (x + y) / 2;
Expression<Func<double, double, double>> newGuess = 
  (guess, x) => average(guess, x / guess);
It will not work, compiler will say: "'average' is a 'variable' but is used like a 'method'". I believe that C# is missing some sugar here, does anyone know why ? And there is quick and ugly solution to the problem:
Expression<Func<double, double, double>> average = (y, x) => (x + y) / 2;
Expression<Func<double, double, double>> newGuess =
  (guess, x) =>  average.Compile().Invoke(guess, x / guess);
I implemented smart trick for expression folding. Let's have static helper class ExComp and this folding method Fold() with this signature:
public static TResult Fold<T1, TResult>(Expression<Func<T1, TResult>> expr, T1 param1)
Of course it could be implemented with Compile().Invoke(), but instead I'm post-processing the expression tree to get rid of method call and folding inner tree in place using InvocationExpression. So whole usage looks like this:
Expression<Func<double, double>> abs = (a) => a < 0 ? -a : a;
Expression<Func<double, double, double>> distance = 
    (a, b) => ExComp.Fold(abs, a - b);
double dist = distance.Process().Compile().Invoke(9,1);
And the tree after processing looks like this:
(a, b) => Invoke(a => IIF((a < 0), -a, a),(a - b))
Next we will hit problem with recursion, which works perfectly for delegates. This is factorial as simpler example of it :
Func<int, int> fact = null;
fact = (y) => y > 1 ? (y) * fact(y - 1) : 1;
Now serious troubles starting with expression trees. Emphasis on TREES. So, there is no good way how to turn expression tree into oriented graph of expressions. Only thing you could do is to use some external variable which has pointer back to root of the tree. For this I'm using similar trick as with folding above, fake method Self() and post-processing.

Back to the Lisp lecture. Now there is introduced idea of higher-order function. The function which creates and returns function, or could compose given function and return composition. Now when we have friendly expression folding, it's a piece of cake.
Expression<Func<double, double>> abs = (a) => a < 0 ? -a : a;
//folding function
Expression<Func<
    Expression<Func<double, double>>, //absFn
    Expression<Func<double, double, double>> //resultFn
    >> foldFn = ( absFn ) => ( a, b ) => ExComp.Fold(absFn, a - b);
//we are passing abs to get abs(a-b)
Expression<Func<double, double, double>> 
  distance = foldFn.Process().Compile().Invoke(abs);
double dist = distance.Compile().Invoke(9, 25);
So now we have foundation for completing square root function using reusable functions.
Expression<Func<double, double, double>> average = (a, b) => (a + b) / 2;
Expression<Func<double, double>> abs = (a) => a < 0 ? -a : a;
Expression<Func<double, double, double>> distance = (a, b) => ExComp.Fold(abs, a - b);

Expression<Func<
    Expression<Func<double, double, double>>, //stepFn
    Expression<Func<double, double, double>> //resultFn
>> avgDamp = (stepFn) =>
    (old, data) =>
        ExComp.Fold(average, old, ExComp.Fold(stepFn, old, data));

Expression<Func<
    Expression<Func<double, double>>, //testFn
    Expression<Func<double, double, double, bool>> //resultFn
>> test = (testFn) =>
    (guess, data, tolerance) =>
        ExComp.Fold(distance, ExComp.Fold(testFn, guess), data) < tolerance;

Expression<Func<double, double, double, double>> sqrtFixedPoint = null;

Expression<Func<
    Expression<Func<double, double, double, bool>>, //goodEnough
    Expression<Func<double, double, double>>, //newGuess
    Expression<Func<double, double, double, double>> //resultFn
>> fixedPoint =
   (goodEnough, newGuess) =>
        (oldGuess, data, tolerance) =>
                (ExComp.Fold(goodEnough, oldGuess, data, tolerance)) ? oldGuess
                    : ExComp.Self(sqrtFixedPoint,
                          ExComp.Fold(newGuess, oldGuess, data), data, tolerance);


Expression<Func<double, double>> square = a => a * a;
Expression<Func<double, double, double>> step = (old, data) => data / old;

sqrtFixedPoint =
    fixedPoint.Process().Compile().Invoke(
        test.Process().Compile().Invoke(square),
        avgDamp.Process().Compile().Invoke(step)
    );


Expression<Func<double, double>> squareRoot = 
  (x) => ExComp.Fold(sqrtFixedPoint, 1, x, 0.000000000000001);

double root = squareRoot.Invoke(25);
Code as well as post-processing helper class could be downloaded here

Thursday, February 21, 2008

Lisp lectures

lisp lectures
In the last months I'm thinking about dynamic languages and trying to understand what's the good part of it. Driven by this curiosity I've came to Lisp lectures by Abelson and Sussman. Yesterday I watched first lecture which is introduction to prefix notation, expressions and functional concept.
As a exercise they model and implement function for computation of square root. My attention was attracted by functional concept, so I tried to implement it. But I was too lazy to get and install any Lisp. Instead of it I implemented it in C# 3.0 using lambda expressions.
I'm curious how far I could get with this approach and whether it will be possible to convert most of the concepts of Lisp and do them in C#. I guess that I will hit the barrier as soon as more dynamic features of Lisp will be shown in next lectures. I will see.
Func<double, double, double> average = (y, x) => (x + y) / 2;
Func<double, double> square = x => x * x;
Func<double, double> abs = (x) => x < 0 ? -x : x;
Func<double, double, double> newGuess = 
  (guess, x) => average(guess, x / guess);
Func<double, double, double, bool> goodEnough = 
  (guess, x, tolerance) => (abs(square(guess) - x) < tolerance);
Func<double, double, double, double> tryGuess = null;
tryGuess = ( guess, x, tolerance ) => 
  goodEnough(guess, x, tolerance)
    ? guess
    : tryGuess(newGuess(guess, x), x, tolerance);
Func<double, double> squareRoot = ( x ) => tryGuess(1, x, 0.001);

double root = squareRoot(2);

Pragmatic programmer

I've finished reading Pragmatic Programmer, book from Andrew Hunt and David Thomas. The book is non-technical, philosophical attitude to programming as a craft. It is full of wisdom and knowledge about how to do your work to keep productive and happy. There are also grains about project management and some motivating articles.
It was published in 1999. I wish I had had chance to read the book those days instead of learning some parts of it the harder way.

Tuesday, January 15, 2008

.NET 2.0 plug-in for 1.1 application is nonsense

Quite evident right ? NET 1.1 application will load 1.1 CLR, which is unable to (dynamically) load 2.0 assembly of plug-in.

Actually I have spent many hours to figure it out. I was confused by too many degrees of freedom in area where my problem could be. I'm playing with IKVM to translate Robocode for .NET (I will post about it later). So I was fully focused on problems in Robocode, IKVM and Java, because all three things are new to me.

Now imagine following behavior:
1) You have some core application (robocode) and you are creating new plug-in (Winforms UI). When you start the core application and specify name of plug-in, it will fail to load it.
2) When you create wrapper/starter application to load core application and call main() on it, UI assembly will load and work. Weird?

Resolution:
I didn't noticed that IKVM 0.36 compiler generates .NET 1.1 core application from robocode.jar for me. This is explanation why first test failed. My .NET wrapper application was compiled for 2.0, so it loaded 2.0 CLR and it was able to load both 1.1 core application and 2.0 UI plugin.

I was unable to see it until Jeroen Frijters told me that I'm using 1.1 compiler. This is not a problem as ikvm 0.37 will be distributed as .NET 2.0 application. I was very happy that the solution was that simple, so it made my day today!