SwingGL - Swing on top of OpenGL

[raft]

hello,

for a few days i've been working on a crazy idea: making Swing work on top of OpenGL. which i think will be very useful if we make it run smoothly. so i've downloaded the jdk source and started digging into it

i first tried rendering, extended awt.Graphics2D and implemented some methods of it. luckily swing seems to use only a small portion of that methods and most can be easily implemented with OpenGL commands. there are some differences though. anyway, below screenshot will give an idea.. the main trick is making top level component believe that it's placed in a container and is visible. than it happily painted itself and its children on given GL based Graphics implementation

second i tried forwarding mouse events from LWJGL display to Swing. this one was hard. all major classes and methods in this process are package private. i still couldnt find a 'normal way' but made a nasty hack: set a private field of some class with reflection. this will almost certainly wont work on a non-Sun JVM and will possibly break in the future. but still proofs, with a proper interleave point making swing event system work offline is possible. i believe, with implementing a custom awt.Tookit and a dummy peer of some component, making event system work without reflection hack is possible. i just couldnt figure out how. maybe some Swing/AWT guru may help ?

later turn will come to keyboard events and keyboard focus. similarly i believe this would be possible with customizing toolkit..

i'm not sure i will have time and continue this project. the code is a mess now, i will publish it when i clean it up. maybe somene will continue it ? if anyone is interested, i will happily go into details..

here is a a web startable link. i tried it on Ubuntu Hardy and Win XP, it worked ok altough there are some small rendering differences. i especially wonder if it will work ok on MacOSX

below is a screenshot, SwingGL running on HelloWorld example (GL one)


cheers 
r a f t

edit: SwingGL can be found at http://code.google.com/p/swing-gl/

[.jayderyu]

That is so darn cool. To me ultimatly the best gui implementation as of yet.

I've been personally thinking about trying to JPCT image rendered into a JavaFX Stage, that way I could then just layer gui items on top. Though I also like Swing for being more a robust gui still.

[paulscode]

I'd be happy to take on this project when the source code is cleaned up.  I've been trying to figure out a way to Swing on top of OpenGL, but I had nearly given up hope until now.

[JavaMan]

That is so darn cool. To me ultimatly the best gui implementation as of yet.

I've been personally thinking about trying to JPCT image rendered into a JavaFX Stage, that way I could then just layer gui items on top. Though I also like Swing for being more a robust gui still.

Not to go off topic, but I found what you said .jayderyu interesting. I have tried rendering jpct in JavaFX. I tried adding a JPanel to a Scene, but that didn't work. Do you know how to render into a Stage? I think rendering into a java.awt.image.BufferedImage and then creating a javafx.scene.image.Image from that might work, but I would like rendering to a Stage as well.


Also, that is a great idea to render Swing onto GL! FENGUI is a little hard on my eyes, at least the uses I have seen.

[EgonOlsen]

Pretty cool. Not that going to use Swing for anything, but still pretty cool...

[raft]

thank you guys well, i love Swing, karga's gui is swing so trying swing on top of GL seems very natural to me. even if we couldnt find a way for event dispatching in a non-hacking way, layouts and rendering are handy things by theirselves, i guess they will make things much easier..

@paulscode i'm glad you're interested in taking on project i'll soon cleanup and publish code. maybe we should setup an open source project at Google code or somewhere ?

[paulscode]

maybe we should setup an open source project at Google code or somewhere ?

Ok, sure.  I've never done any kind of collaborative open source project before, but I reckon there's a first time for everything.

[raft]

me neither but i guess using SVN will be sufficient
at the moment SwingGL uses jPCT's FrameBuffer to blit images. i've got some GL help from Egon and implementing a pure GL blit. than i will setup a project a google code

[raft]

ok, i've cleaned up the code and setup a Google-code project: http://code.google.com/p/swing-gl/
it's somewhat funny indeed, publishing a few classses as an open source project 

all source and a demo can be found there. demo still uses jPCT but rendering is completely on top of LWJGL now. i've also corrected some minor color things, we get a cleaner render now.

any contributions are welcome.

paulscode, i've sent you a pm but possibly you didnt see yet

[paulscode]

Ok, I sent you my gmail address.  I'll go look at the code when I get a chance.

[EgonOlsen]

Cool. About the RGBA-issue, that you've written in your mail: LWJGL is fine here, it's just that the ordering is in little endian, which can sometimes be a bit confusing. If you read your color entries from the right to the left, they are perfectly RGBA...they just look like ABGR...

[raft]

ehm, i easily got lost in that one zero labyrinth

[paulscode]

This thread has been dead for a while, because I kind of put the project on a back burner after looking at the code (I have very little experience with OpenGL or Swing, so I was having trouble understanding it well enough to really contribute).  That being said, I want to look into this project again.  My recent work with using ITextureEffect to draw from an awt.Image context onto a texture got me thinking that this could potentially be used to create a common interface for displaying Swing on top of a rendered scene, in either hardware or software rendering mode (by drawing the components onto an Image and then drawing that onto a texture used by an overlay, for example).  I'll start playing around with the code some more and post on various forums to try and understand it a little better.

[raft]

good to hear that

[majitri]

hello, how can see the code for your project?