Sometimes I write things, sometimes I don't.

To content | To menu | To search

Wednesday 18 September 2013

GIR for java-gnome: last update / summary

Continue reading...

Wednesday 11 September 2013

GIR for java-gnome: update week 36

There is not a lot of things to talk about for the past week.

The code generator can now be considered as stable and I decided to cover the GWeather library to demonstrate how to add the coverage of a new library. You can check the progress of this work in this branch. There are still some examples and tests to write. I will also write a quick how-to guide to help the contributors to implement a new library.

See you next week for an shorter report too? No it will be the last one so I will do a recap of all the work that I have done during this great Google Summer of Code.

Wednesday 4 September 2013

GIR for java-gnome: update week 35

I am a little late on this report due to the end of the holidays and the start of a new university year.

Only 3 weeks left for this GSoC so it is time to polish everything and document the work that has been done. This is why the week 35 has seen the latest big changes in the code so I can now focus on details. There was polishing of course such as code optimization but also pretty big changes such as the followings:

  • All the GIR data are now loaded in memory before the parsing. Actually the code now starts to parse all the XML data and keep references to the whole <repository> elements in an IntrospectionRepository object. This object contains references to <namespace> elements and headers files to include in the generated C code. A list of C identifier prefixes is also built when scanning all the repositories. The IntrospectionParser is now fed with IntrospectionRepository objects and parse each namespace contained in each repository. This change allowed to remove the hard coded list of C identifiers prefixes which make the code more flexible.
  • The combined whitelist and blacklist format for the types.list file has been changed to XML. Since that we know how to parse and use XML data properly in java-gnome now, it was a obvious step to take. The XML based format is a lot more flexible than the previous format and allows us to include more information in it. A small summary of the format to respect is provided at the beginning of the file so the contributors can easily understand how it works.
  • Due to the change of format for the types.list file it was possible to include the Java classes names overrides and the Java packages names overrides in the same file. So now the build is depending on only one file which is a lot better when you want to add, remove or change something in java-gnome.
  • To prevent the java-gnome build process from failing, The org.gnome.glib.File class was made public. This change was not meant to be definitive since we do not use this File outside the package. So I changed this class scope back to default and I added to the blacklisted the functions that were relying on it. We have a well working File class in Java we do not need another one.
  • The rest of the work was polishing, removing useless code, updating comments and email writing.

Week 36 and 37 goal is to add the coverage of another library thanks to the new Introspection based parser and write a simple how-to about it. After that the GSoC will be almost over.

Tuesday 27 August 2013

GIR for java-gnome: update week 34

The Google Summer of Code is ending in a month and it is time to make things stable, to do some polishing and to write documentation.

The goal of the week 34 was to improve some parts of the code that I have done so far to make them less cumbersome and more maintainable. I have started with a couple of code cleanup and by adding some comments where they were needed. After that I have worked on a piece of code that bothered me for some time now : guessing the C type to give to the code generator based for a return value or parameter. You may remember the ugly method that I have written about 3 weeks ago. It is now removed to be replaced by two methods which do the work in a better way starting from this commit. The idea behind these two methods is: taking an XML element (a return-value or a parameter element) and scanning it to find a type and the modifiers applied to it (const for example) and then generate a string that our code generator can understand (const-GList-GtkWindow* for example). This code is still depending on an hard-coded list of modules that we handle. I need to modify this to make it dynamic and retrieve modules' C identifier prefixes with the c:identifier-prefixes attribute of a namespace element.

The second part of the week has been used to write test cases to validate that the Introspection parser is working properly. It tests that we are creating Block elements properly and also that they contain the needed data to generate the code without any troubles. To make this possible I had to change the input of the IntrospectionParser to make it able to parse files but also strings.

Eventually, I have dedicated the end of the week to modify the output of the IntrospectionParser. Starting from this commit the parser now outputs a map of Block arrays which are identified by strings. I have made the following change to change the behavior of the code that override or add data thanks to the .defs files. This operation is now done right from the BindingGenerator class instead of doing the Block change inside the DefsFile class. I have made this choice because I believed that a DefsFile object should not be modified after being created. So now we have a code generator working like this:

Like a said last week, java-gnome using GIR data is now compiling and passing all the tests suite without any problems. There is still some code to improve to be able to remove some ugly hard coded things such as headers files to import (we probably need a keyword in the types.list file for example). For the end of this GSoC I plan to add the coverage of another library and to write a how-to for the interested (or future) java-gnome's hackers.

Monday 19 August 2013

GIR for java-gnome: update week 33

I am glad to say that I am ahead of my schedule. The GSoC will still continue for approximately a month but I have already reached the goal that I was supposed to reach in 2 weeks. So I am happy to say that since this commit java-gnome compiles and passes the tests suite with the Introspection based code generator. There is still work to do though. So the plan is to focus on the code polishing and the documentation of the new code generator behavior. I have written a mail on the java-gnome hackers mailing list to inform and summarize my work to the contributors. There are also several questions that still need answer to make sure that everyone will be happy with the new code generator.

I have done a quick graph to show how the code generator works in my java-gnome Introspection branch.

As we can see the data that feed the code generator come from 2 sources. The first and main source is the GIR data and the second source is the DEFS data. We are using DEFS data for several years in java-gnome and they are now an optional source of data because GIR is meant to replace DEFS. Why did I choose to keep .defs files? To be able to add coverage for libraries that do not provide GIR data yet or to override methods, functions and signals. In this way we can combine the accuracy of GIR data with the power of the .defs parser. Some people might be confused when viewing the DefsFile[] structure in the middle of the graph. This is not a step that says that we are writing .defs files from GIR data. DefsFile is actually a class that represents what we parsed before. Until the Introspection it was only DEFS data but now I guess that this class needs a better name.

Here is a list of what I have done during the week 33:
  • Handling of GError** parameters when functions throw error.
  • Providing a new way to whitelist types and blacklist methods, functions and signals.
  • Cleanup some part of the code.
  • Add several special cases where constructors and methods needed to be overridden.
  • Fix some unit tests.
  • Fix the parsing of enumerations.
  • Fix some memory management issues due to constructor return value owner problem.
Things are now getting stable, the goal to the end of this GSoC is to polish the code, write some documentation, take some decisions to validate or invalidate some design choices, add the coverage of another library (WebKitGTK+ maybe, if you have an idea feel free to tell me) and in the future, release java-gnome 4.2.0 with a fully Introspection based bindings!

Tuesday 13 August 2013

GIR for java-gnome: update week 32

The status of the java-gnome compilation process of last week reported about 250 errors. There are now 6 errors left to fix. That is a pretty good progress that I have done during the week. The commit that I just did few minutes ago summarize the work that I have done quite well.

So how is the code generate fed in my java-gnome "introspection" branch?

  1. First it needs to pass the configure step. This step will check that the .gir files (containing XML data) are available on the system. Andrew is working on a patch so we can give a --gir=/directory/containing/gir/files to the Perl configure script.
  2. The second step of the process takes all .gir files and parse them thanks to the IntrospectionParser class. The parser loads a whitelist of types that we can use to avoid the parsing and code generation for types that we don't need (GRand for example).
  3. The following step will take .defs data that can be found in the src/overriders directory. This .defs data will be used by the code generator to add data or to redefine the data take from the Introspection files.
  4. Once all the previous steps are done, the translation layer and JNI layer are generated.
  5. Finally the make process try to build everything and this is at this time that we can see if the public API matches with generated code.
To be able to do all of this process I had to modify several part of the existing classes by adding methods to get information about a Block object (the corresponding C name, the object it belongs to and if it represents a constructor or not. A Block is a data structure that the code generator uses to represents a type, a method, a constructor or anything else.

The XML parser still needs some polishing to handle the throws attribute and to make it more beautiful to read and maintain.

Sunday I had a meeting with Andrew to talk about my progress and what we are going to do next. We agreed that it would be interesting to start merging some code into java-gnome official repository. We also talked about having regular Google Hangout so we can continue to discuss. Since the progress on using Introspection data to fed the code generator is quite good, we talked about trying to add coverage for an another GNOME library to add some value to the work that I have done so far.

Wednesday 7 August 2013

GIR for java-gnome: update week 31

The status update of my work is a little late this time due to some personal issues I had this week end.

The work is mostly the same as during week 30. The XML parser needs improvements a lot I found improvements to make every day. So now the parser can handle Union types, the special methods like "free" and "copy". I also had to tweak the public API of java-gnome due to some changes introduced by the use of Introspection data (some variables to delete because they do not exist anymore for example). Since we will probably do a major release when the Introspection based code generator will work I guess that all the public API changes will not be a problem.

The other part of my work consisted in trying to make the Introspection parser more maintainable. It has grown a lot since the beginning of this GSoC and sadly it has become less and less readable and maintainable due to copy/paste of code blocks etc… Currently, the Introspection parser only is composed of about 1800 lines. To compare, the .defs files parser is composed of about 390 lines so there is a big difference. I moved some lines of code into static methods so they can be used more easily and be more maintainable.

The code of the XML parser is starting to get stable because it works in most of the cases. I still have to investigate about 250 compilations errors when building java-gnome (errors produced by the files that the code generator writes). A lot of errors are due to the way the .defs files were written before so they should be fix in the following weeks.

I had to do some black magic to bypass some problems with GList and GSList parameters or return types and also with parameters where the name was specified but not the real type. I do not know if it is something generated on purpose when building the Introspection data or if it is a bug but it happens sometimes. Lets take an example. A common seen XML description of a parameter in a .gir file looks like this:

<parameter name="pixbuf" transfer-ownership="none" allow-none="1">
 <doc xml:whitespace="preserve">a #GdkPixbuf, or %NULL</doc>
 <type name="GdkPixbuf.Pixbuf" c:type="GdkPixbuf*"/>

But sometimes we can found this:

<parameter name="pixbuf" transfer-ownership="none" allow-none="1">
 <doc xml:whitespace="preserve">a #GdkPixbuf, or %NULL</doc>
 <type name="GdkPixbuf.Pixbuf"/>

So this is a tricky part because of several things:

  • The code generator can not handle a type like GdkPixbuf.Pixbuf it actually needs GdkPixbuf.
  • How can we know if it just a GdkPixbuf or a GdkPixbuf* ?
  • If we have just EventType how can we know what namespace owns EventType (Gdk, Gtk, ...)?
So here is the black magic. Can we make thing uglier? :(

A meeting with Serkan Kaba (my mentor) and Andrew Cowie (java-gnome's maintainer) has been set up to discuss about my progress and to make some technical choices to decide what path we should take.

During this week (week 32) and the following I will try to implement a way to override Introspection information so we can control the behavior of the code generator like we can when using the .defs files.

Tuesday 30 July 2013

GIR for java-gnome: update week 30

Not a lot of commits for this week but there was work. The only commit I have made was to detect dynamically the location of Introspection XML files for each distribution (for now only Debian based distros are handled).

Most of the work that I have made is not commited yet. I worked on the XML parser to:

  • handle both virtual methods and GLib signals,
  • handle functions,
  • handle fields for boxeds.

There are still work to do to have a proper parser that can handle without any problems all the .gir files. Each time I fix something two new problems appear but heh this is how computing stuff works. The XML parser is getting larger every day and I will have to refactor it once more to reduce the amount of code due to some copies and pastes. The Introspection files are now detected based on the path where they are put by the distributions packages. This is done at the ./configure step. The script tries to find the files and asks the user to install the required package if it fails.

I have also planned to provide a way to override Introspection data to make the code generator behaves like we want to. This is a little bit the case right now with the possibility to override the names of objects but I would like to extend this feature to the whole range of things that we can handle (methods, functions, etc).

In my proposal I have written that I would like to have a code generator using Introspection data that would work with a minimal sets of types. It is the case right now. The code generator works, it generates JNI and translation layers, but there are still errors on some parts. For a basic class the code generator is working really well but when the complexity (in the Introspection side or even in the java-gnome side) is increasing problems appear. One of the problems right now is the handling of parameters and return values based on GList and GSList for example. I hope to fix all of this soon to continue with more interesting stuff.

- page 1 of 2