Monday, September 8, 2008

Debug Listeners on AWT/Swing components

Being a newbie to the whole Swing stuff, I often wonder what events are fired on a given component for any given gesture or event.

I wonder no more!

Copy the following (ctrl-C), then activate some package node in a project in Eclipse (e.g. the obvious "com.example") and paste (ctrl-V).

import java.awt.Component;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import javax.swing.JButton;

public class DebugStatics {

public interface LineTaker {
void line(String msg);
}

public static final void main(String[] args) {
LineTaker out = new LineTaker() {
@Override
public void line(String msg) {
System.out.println(msg);
}
};
addDebugListeners(out, new JButton(), false);
}

public static void addDebugListeners(final LineTaker out, final Object component,
final boolean includeAllMoveEvents) {

// :: Find all add*Listener methods for supplied component.

List<Class<?>> listenerInterfaces = new ArrayList<Class<?>>();
List<Method> addListenerMethods = new ArrayList<Method>();
Method[] methods = component.getClass().getMethods();
for (Method method : methods) {
String name = method.getName();
if (name.startsWith("add") && name.endsWith("Listener")) {
Class<?>[] parameters = method.getParameterTypes();
if (parameters.length == 1) {
Class<?> interfaze = parameters[0];
if (interfaze.isInterface()) {
listenerInterfaces.add(interfaze);
addListenerMethods.add(method);
}
}
}
}

// :: Make handler for "super listener"

InvocationHandler handler = new InvocationHandler() {
long _lastEvent;
String _lastMethodName;
boolean _threadFired;
InvocationHandler _handler = this;

@Override
public synchronized Object invoke(@SuppressWarnings ("unused") Object proxy, Method method, Object[] args)
throws Throwable {
_lastEvent = System.currentTimeMillis();
String methodName = method.getName();
if (includeAllMoveEvents || !(methodName.endsWith("Moved") && methodName.equals(_lastMethodName))) {
out.line("event:[" + method.getName() + "] - on - [" + method.getDeclaringClass().getName()
+ "] - with - " + Arrays.asList(args) + ".");
if (!_threadFired) {
_threadFired = true;
new Thread("debug:Event Stream Breaker") {
@Override
public void run() {
while (true) {
long millisLeft;
synchronized (_handler) {
millisLeft = 400 - (System.currentTimeMillis() - _lastEvent);
if (millisLeft < 0) {
out.line("===== event stream break.");
_threadFired = false;
_lastMethodName = null;
break;
}
}
try {
Thread.sleep(millisLeft + 2);
continue;
}
catch (InterruptedException e) {
break;
}
}
}
}.start();
}
}
_lastMethodName = methodName;
return null;
}
};

// :: Make "super listener", "implementing" all the Listener interfaces

Object superListener = Proxy.newProxyInstance(DebugStatics.class.getClassLoader(), listenerInterfaces
.toArray(new Class<?>[0]), handler);

// :: Attach "super listener" using all add*Listener methods on supplied component

for (Method method : addListenerMethods) {
try {
method.invoke(component, superListener);
out.line(" ++ add*Listener: [" + method + "].");
}
catch (Throwable e) {
out.line("Got error when trying to invoke add*Listener method:[" + method + "]." + e);
}
}
}
}
Code discussion: The LineTaker comes from the need to have some flexibility regarding where the textual stream of events should end up (e.g. to System.out.println(...) or to some logger.debug(...)). The driver (main method) is just to check out the listener adding (and possibly also to find out which types of add*Listener methods a given AWT/Swing/Whatever class has). The reason for the Thread-forking is so that you more easily can understand which events "belongs" to each actual, physical gesture: Move the mouse into a button, wait a second, then press down mouse button, wait a second, release mouse button, wait a second, move out. Now the result event stream have embedded a break on every second-wait. The thread only lives long enough to write the break, then exits. The reason for the code regarding move-events suppression is that you get an awful lot of move-events without it - this code results in only one move event being recorded in a consecutive run of the same move event.

While dumping the code into this post, it hit me that the above method actually works for any class having add*Listener(AnyInterface listener) methods. That's why the type of the component argument is Object.

There's a pretty impressive amount of events being fired on a JButton for the simple act of using the mouse to click it!

Thursday, August 21, 2008

Subversive svn: authentication cancelled

I just wanted to record this for anyone that gets a problem that I had, as I didn't find the solution "by the way of Google". Do note that I'm no expert on either Subversion or Subversive..!

Before doing anything described here, please back up your workspace! I'm not talking about fetching that monster tape drive, install and run some huge backup software and then ship the tapes off to another continent - just make a copy somwhere of your Eclipse workspace. This so that you can start over, trying another angle, when things doesn't work out and your whole workspace is accidentially encrypted using your lavalamp's current formation as the key! (You then shut down Eclipse, delete your workspace, and "restore" the backup by copying it back. Start eclipse, and try your new approach.)

Before going further, first just attempt to start Eclipse with the "-clean" argument. You should not even need to worry about that backup then - and it might fix everything.

I have recently upgraded Eclipse to 3.4.0. I backed up my Workspace to some temp space, and then just pointed the new Eclipse to the same Workspace (the existing, "old" copy..). This magically worked nearly flawlessly: The Workbench looked exactly the same, all views set up the same, all Mylyn local tasks (which was the only type I had) was set up with contexts and everything.

But there were apparently some problems. The first one I don't quite remember, but Eclipse/Subversive at some point asked me if I wanted to reconnect to "the now-deleted repository location" something "https://blahabla...". And I then also noticed that all the Subversive markings (where you are (trunk), version numbers, colours, markings, whatevers you've configured Subversive to adorn your project with) were gone. I just said yes, and hoped for the best. Nothing happened. But after a reboot of Eclipse (I've found that much fixes itself by booting this application!), everything seemed to work.

But today, I wanted to commit something. This didn't work out at all, and in the Console window, I got "svn: authentication cancelled" and "blahblah Error". The annoying thing was that it just never asked me for credentials, so I couldn't give it my password (the username was saved, at least earlier). When this happened, Eclipse popped up a screen stating that some error had occurred (I don't remember the wording here, it was terse and useless), stopping up.

So, I did a couple of things which apparently seemed to work:
  • Deleted "C:\Documents and Settings\<username>\Application Data\Subversion", based on this Oracle bug. You'll then be asked whether the certificate is correct etc etc upon next attempt to do Team->Update. Do note, however, that I don't think this is a part of the solution! So you may go ahead without doing this, at least the first round..
  • Back up the project in question, in particular if you have heaps of uncommitted stash there! (If you don't have uncommitted data, you might just as well delete the entire project, and just start with a fresh checkout).
  • Right-click on the problematic project -> Team -> Disconnect
  • Choose "Do not delete the SVN meta-information (e.g. .svn subdirectories)", click Yes.
  • After a while, you'll see that the project disconnects, and the Subversive markings disappears.
  • Now, you go Right-click -> Team (Don't freak out by this menu being pretty much empty!) -> Share Project...
  • Pick SVN (Obviously), click Next
  • Pick the "Create location using project settings" (second choice here).
  • You'll come to the "Enter Repository Location Information" page.
  • At this point, the one thing that I immediately noticed, was that it suggested URL "https://blahblah/svn/<projectname>/trunk". I don't think the "trunk" part shall be a part of the location, so I removed it.
  • Check the "Validate Repository Location on finish".
  • Enter your username and password. I don't want to store the password, so I leave "Save password" unchecked.
  • You may at that point click the "Browse" button to check if your URL is good - it should show the "branches", "tags", "trunk" directories.
  • On the Advanced tab, check "Enable Structure Detection".
  • I don't have anything fancy for the SSH Settings nor SSL Settings.
  • When I clicked Finish, the Subversive markings reappeared, everything worked, and I could update and commit right away!
  • Note that the repository location, which you may enable the project name node to be adorned with by Subversive, reads "https://blahblah/svn/<projectname>", leaving out the "trunk" part. The full text on my project name node is "[https://blahblah/svn/<projectname>: trunk]" - but this isn't standard config for Subversive, I believe.
The "trunk" in the URL stuff is actually what I think might have gone wrong the first time around with the first problem described above, where I just accepted what Eclipse suggested to me as solution. So this is furthermore what I suspect has been the reason behind the whole authentication cancelled thing: it really had the wrong URL, but the Subversive client handles this aspect (as it still points to the same place: either, wrongly, "trunk" on the URL and "" on the branch-selection, or more correctly, sans "trunk" on the url, but "trunk" on the branch-selection), but don't any longer know how to handle authentication..

Actually, I have ended up with something rather like this once before: I didn't know what URL Subversive wanted, so I navigated using the browse button, and navigated all the way into the "trunk" directory, as it was trunk I wanted to check out and work with, and I didn't immediately see any other way to select that. Subversive is seemingly quite happy with this, checking out your project and whatnot. But then, later in the game, things start to weird out. I don't quite remember how this manifested itself, but I do remember that I got some strange recursion problem, where I could see the entire trunk/branch/tags structure within my project. When I realized the URL thing, deleted the project and started anew, everything cleared up.

I think Subversive should be nice enough to point out that you've most probably navigated wrongly now, as the latter part of the SVN URL is at this point looks like a "known structural ending", and this should not be a part of the URL blah blah. See, Subversive does know what the standard structure of a SVN setup is - so it could easily have done this. I filed a bug about this (a year or so ago), but so far nothing have happened.

Update 2008-09-08: I got the same problem again some days ago. Subversive doesn't use the normal Preferences screen to define subversion repositories. It has its own view, called "SVN Repositories" (in the same way that Mylyn has "Task Repositories"). Here you can find the different repositories. Right click, and select "Location Properties..." (Not "Show properties", as that refers to subversion properties; Those things that you can stick on directories and files within the repo, e.g. "svn:ignore" @ trunk). Hitting the Location Properties, you get that Edit Repository Location dialog which I refer to in the above hundred or so bullets. If you here click the Browse button, you will get a username/password dialog (given that they aren't correct on the parent dialog). You should be able to browse the repository. When this works, stick in the username/password in the Edit Repository Location dialog again, tick off "Validate Repository Location on Finish", and hit the "Finish" button. The authentication problem is now gone.

In addition, I believe I might still be mistaken about what URL the location should be. I have now changed it to not include the <projectname> part as I stated above, as the URL without that part would be the location to the actual Subversion repository. This seems more logical (and it still works!).

Furthermore, after a couple of those "svn: authentication cancelled" rounds, I tried starting eclipse with the "-clean" argument. I have not had the problem afterwards. Whether it was the location URL, or the -clean that finally fixed it, I am not sure about, although I tend to believe the latter.

Finally, it is actually also possible to completely disconnect from the repository (Team->Disconnect), choosing "Also delete the SVN meta-information from the file system" from the Confirm Disconnect from SVN dialog. You can then Team->Share, make a new repository location, and from that pick the same project you have the files for. Subversive apparently realizes that the project name you try to share it as, already has information in it, and apparently checks it out at a temp location, then copy back the files, seemingly keeping your changes. I would however not bet on that - this really seems just as risky as to start afresh with a new checkout of the project. In any case here, back up your workspace before trying anything!!

Wednesday, March 5, 2008

Debugging and Profiling - same, not different

Why is it that Eclipse has such a great debugger, but such a extremely poor profiler?

I've used JProfiler and YourKit Profiler, and find both products interesting (however, JProfiler gets my money). However, they are a just that slight tad too awkward to fire up - I don't use them often enough, so it is not "in the fingers" in the same way I always run my programs through the Eclipse debugger.

The Eclipse profiler, when I've gotten it to work, which is a real PITA, is amazingly slow - I've never been able to profile even the simplest applications. I wonder if it doesn't use some XML format to transfer the profiling information from the profiled application to the profiler..

The Eclipse debugger, on the other hand, is a truly wonderful tool - for what it is made for.

Then, one suddenly starts to wonder: aren't these two things pretty much the same stuff? Both the debugger and the profiler needs to be able to inspect the objects on heap, control threads, inspect stacks. This has been recognized by the JVM vendors, where the two JVM interfaces, JVMPI (JVM Profiling Interface) and JVMDI (JVM Debugger Interface), have been unified into the JVMTI (JVM Tools Interface). So why aren't simply these two JVM tools indeed a unified power tool?

I've so many times wanted to just stop the application and check out what all the memory is used for right now - "Heap Walking" in JProfiler terminology. Or just do a quick analyze of where the CPU is being used from here on to there on, using for example two breakpoints. There are plenty other scenarios where the features of a debugger and a profiler overlap, and where just one of them doesn't quite cut it. Starting the JVM with both JProfiler and Eclipse attached actually seems to work in some fashion (as long as you don't pause JProfiler's handling threads, as the JProfiler GUI then freezes), but I always seem to get Threads in the debugger that when paused, don't give me the stack, but instead show "", causing Eclipse to pretty much freeze too, and it is hardly the perfect situation at any rate.

Here's some possibilities:

  • Take a heap snapshot at some certain breakpoint.

  • Pausing the application, then taking a "heap walking tour". Resume, then exercise the application by doing that thing that always messes up. Pause again, snapshot, compare difference (or use breakpoints instead of manual pauses).

  • Stepping the application, having one thread in pause. Make a heap snapshot of all objects that the current thread's stack points to, displaying it graphically along with the stack trace, arrows pointing out from each stack element.

  • .. and being able to expand a Thread's stack elements and view the actual variables that hold on to each part of the heap graph, visually using arrows pointing to the objects

  • When in "heap walking", have a more direct understanding of what threads hold on to what objects.

  • Start allocation recording and/or cpu recording (start profiling) when any thread reaches some breakpoint.

  • Do timing analysis (profiling) of a single thread between two breakpoints. Do this repeatedly while the application is being exercised, and aggregate.

  • By default pause the application at a OutOfMemoryError and do a heap snapshot.

  • (Admittedly only a profiling wish:) Like YourKit (unlike JProfiler), let heap snapshots be "full" (include the data) and standalone, with thread stack information, in that the entire heap is dumped to disk, so that it is possible to easily compare.


Basically - what I want is the integration of the features that a debugger has, and what a profiler has. In particular the combination of stack-inspection from the debugger with heap-inspection from the profiler, and the code-integrated features with breakpoints and exception-breakpoints that a debugger has (the now standard fully integrated IDE debugger), triggering the profiling and heap snapshotting features that a profiler typically has.

I think that if these JVM inspection tools wasn't made with one or the other application in mind but instead thought of as one unified unit, a much more powerful tool would emerge. (I nearly get that tech-drool when thinking of how cool it would be: "so, I believe this object should go out of scope at this breakpoint.. So, why isn't it GC-able now? Lets trace-to its GC roots..").

Now, I have understood that Netbeans have a much more integrated profiler than Eclipse. I really should check Netbeans out someday very soon..!

Monday, February 25, 2008

Too much verbosity in generics!

Why do I have to write the following bold part:
Map<SomeLongClass, SomeOtherThing<Another>> myPoint = new HashMap<SomeLongClass, SomeOtherThing<Another>>();

(Update 2009-03-03: As you probably already knew, I am not the only one realizing this!)

As far as I know, anything else than a verbatim copy of the stash that already is present in the declaration of that variable/field is a compile error. So why do I have to type it?? Argh..!

And OH so annoying Eclipse behaves if you haven't yet imported Map while writing that line - because if you've come to the point where you're about to write "Hash.." and then think that it would be reeeally nice if Eclipse could just autocomplete the rest of that shit for you, it doesn't quite work! And at that point it doesn't help to arrange imports either, because the code is currently to uncompilable for poor Eclipse to even understand what you're aiming at, so you'll have to delete up to the equals-sign, stash in a semicolon, then arrange imports, and finally start up with the "Hash..." again, now having autocomplete functionality at the ready. Grr..

Please, someone: I'd like Java itself to remove the need for that duplicated stupidity, and then Eclipse to be better at trying to compile partially if it doesn't understand the line the cursor is on (so that at least arrange imports would work at the described annoyance), and finally, I'd like some extra-super context-assist for writing the line in the first place - letting Eclipse know of the standard implementations of all Collections, and also picking up the classpath. For that matter, it could do such stuff all the time.

(And while I'm whining: Blogger doesn't handle <'s and >'s in "rich text" mode..)

Thursday, February 21, 2008

Java Garbage Collection details - "Garbage First" G1

For folks into the details of java, there's an interesting blog entry called "Our Collectors" over at Jon Masamitsu's blog, who apparently is a Sun GC guru. It has a nice low-down over the different collectors - what they are, where they are used, and how they can be combined.

Also, this is a good place for some insight into the G1 GC - "Garbage First", a new algorithm that is going to be the default in Java 7.

Some interesting discussion ensues, where Tony Printezis, apparently another Sun insider, chips in with comments.

http://blogs.sun.com/jonthecollector/entry/our_collectors

Also, don't miss this good whitepaper, Memory Management in the Java HotSpot™ Virtual Machine (pdf, April 2006).

Friday, February 15, 2008

Linux RAID1: Failing disk in boot set, part 1

I'm running software RAID on Linux, using the md module.

I have everything RAIDed, including /boot and swap.

The /boot and / is on a RAID1 set. The logic behind that is the boostrap problem: the kernel must be loaded to enable the md_mod module and hence enable RAID, but the kernel resides on the RAID set - catch 22! However, since RAID1 is pure mirroring, things work out: assume the boot disk isn't a part of a RAID set while loading the kernel from /dev/sda1 in read-only mode, then, when it comes up, load and activate the RAID module, enable the RAID set, and continue taking up the system using the now /dev/md0 device.

So, if a disk fails, find out which one (find its serial number), chuck that out, stick in a new one, and then boot up. The RAID module will upon trying to assemble the RAID sets discover that one of the device is gone, and simply take up the set in reduced mode. You are then able to partition your new disk exactly as the one already present, and then add the new device back into the RAID set.

Simple!

However, there is a bootstrap problem before the one described, which is what really is called "the boot process": When the computer starts up, it reads the first sector of the boot harddisk (typically the first in its discovery sequence, possibly changeable in the BIOS setup) into memory, and runs that. This is where the bootloader resides.

The only problem is that a normal system only sticks the bootloader on the mentioned boot harddisk. This part of the disk isn't handled by the RAID module - so it won't be copied over to the other disk (e.g. the partition table is there too - can't have two different sized disk's partition table hard-copied from one to the other).

So, if a disk fails, you'd better hope that it is the other disk, not the boot disk!

I wasn't that lucky...!