People have been asking how to leverage models on and off on the web and in meetings I attended, so I thought I might share this little tutorial with you. In this tutorial, we will develop a little code generator that helps you to create (HTML) forms from models.

A short Overview

Xpand is a template engine, similar to FreeMarker, Velocity, JET and JSP. However, it features some very unique properties that makes using Xpand very well suited for generating code from models, such as type safety and polymorphic dispatch. If you haven't heard those terms before, fear not! I'll show you how to use Xpand by way of an easy-to-follow example.

The usual process of writing a code generator with Xpand is as follows:

1. Define the structure of the model you want to process. This is called a metamodel
2. Define one or more template(s) that teach the code generator how to translate your model into code.

Easy, isn't it?

Using the code generator is even easier:

1. Create a model
2. Start the code generator

It is worth mentioning that you can use Xpand to generate code for almost any known programming language. Everything you can express in text can be generated using Xpand. So, while we will be generating HTML and Java code in the following example, you can easily write code templates that generate code for C#, Basic, Lua, SmallTalk, ABAP, or any other programming language. You can also generate manuals and other documentation artifacts from your models using Xpand. I've successfully used Xpand to create DocBook files from models. Those DocBook files have then been converted to PDF files and online help files.

Preparing your IDE

1. Grab and install a recent copy of Eclipse. At the time of writing, I am using Eclipse 3.6 M6.
2. Install the latest version of Xpand (Help -> Install New Software ...)
3. Add the Xpand update site (at the time of this writing, I am using Xpand 1.0 nightly builds from http://download.eclipse.org/modeling/m2t/xpand/updates/nightly/)
4. Add the MWE update site (http://download.eclipse.org/modeling/emft/mwe/updates/nightly/)
5. Select MWE and Xpand:
   
6. After the obligatory restart, do yourself the favour and set the platform encoding to UTF-8!

Creating a Generator Project

Xpand code generators are hosted in Eclipse plug-ins, mainly because this makes handling the classpath a lot easier. People have reportedly used Maven to run Xpand code generators, but we won't go down this road today. Let's create a simple generator project:

1. Open the new project wizard and choose Xpand Project from the list
2. Choose a meaningful name for your project (e.g. org.xpand.example.gettingstarted
3. Select Create a sample EMF based Xpand project

After clicking Finish, the wizard will create an sample generator project for you:

Before we can start working with this project, we need to perform some clean-up actions:

1. Open src/metamodel/Checks.chk and delete all its contents
2. Open src/metamodel/Extensions.chk and delete all its contents
3. Open src/template/GeneratorExtensions.ext and delete all its contents
4. Delete src/Model.xmi
5. Don't forget to save all modified files

Creating the Metamodel

As mentioned before, we need to define the structure of our models before we can actually start writing the code template.
Xpand is capable of understanding a variety of metamodel types. For example, if you have an XML schema file, you can use this as a metamodel and thereby enable Xpand to use XML files which are compliant to your schema as input models. Or, if you already have a bunch of Java files making up your data model, you can use those to drive Xpand code generation. In this tutorial, however, we will be using an Ecore metamodel to define the structure of our models. The project has already been configured to support Ecore metamodels, so all we need to do is open metamodel.ecore and define the structure:

1. Please open src/metamodel/metamodel.ecore

.

2. Remove the following elements from the metamodel: Feature, Entity, Datatype, Type, Model. The metamodel should now be empty:
   
3. Select package metamodel (as depicted in the screenshot above) and add a new EClass (context menu -> New Child -> EClass). Use the properties view to change the name of the newly created EClass to Model.
4. Create a new EClass Form
5. Select the newly created EClass Form and add the following EAttributes:
   • name, set the EType to EString
   • description, EType = EString
   • title, EType = EString
6. Select EClass Model and add a new EReference, setting its attributes as follows:
   • name = forms
   • EType = Form
   • Containment = true
   • Upperbound = -1 (meaning: unlimited)
7. Create a new EClass Field and add the following EAttributes to it:
   • name, EType = EString
   • label, EType = Estring
8. Create another EClass TextField, setting its properties as follows:
   • name = TextField
   • ESuper Types = Field
9. Add one EAttribute text to Textfield:
   • name = text, EType = EString
10. Add an EClass MultiLineTextField to the metamodel:
    • name = MultiLineTextTield
    • ESuper Types = TextField
11. Now that we have everything in place, we finally need to add a reference from Form to Field so we can later add fields to a form. Select EClass Form and add an EReference to it, setting its properties as follows:
    • name = fields
    • EType = Field
    • Containment = true
    • Upperbound = -1 (meaning: unlimited)

Creating a Model

Let's now create a model that follows the structure of the metamodel:

1. In metamodel.ecore, select EClass Model
2. Create a model instance by choosing Create Dynamic Instance... from the context menu
3. Save the model file to src/Model.xmi

The model file editor will now open and you can use the tree editor to input the following model:

• Add a Form to the model, seeting the following properties:
  • Name: context
  • Description: Send your feedback
  • Title: Contact form
• Add a TextField to the Form, setting the following properties:
  • Name: name
  • Label: Name
• Add another TextField to the Form, setting the following properties:
  • Name: email
  • Label: EMail
• Add a MultiLineTextField to the Form, setting the following properties:
  • Name: message
  • Label: Message

Your model should now look like this:

Creating a Code Generator

As mentioned before, we will create a code generator for simple forms. Nothing too fancy, but enough to give you an idea of how to create generator templates.

The result will look like this:

Open src/template/Template.xpt and replace its contents with the following text:

«IMPORT metamodel»
«DEFINE main FOR Model»
«EXPAND form FOREACH forms»
«ENDDEFINE»

On the first line, we import the metamodel so that the generator (and the editor as well) knows about the structure of our model. On line 2 - 4 we define a code template named main, making sure it is bound to model elements of type Model. The template doesn't do much, except to call another template (which we will define in a minute) named form with the collection of Forms, contained in the reference forms of the current form.

Add the following lines to the template file, defining the template for the HTML file:

«DEFINE form FOR Form»
«FILE name + ".html"»



  
  

In the first line, we start the template by specifying its name (form) and the type it is bound to (Form). On the next line, we use the FILE statement to specify the file the output is going to be written to. The name of the file is derived by concatenating the attribute name of the current Form and the string literal ".html".

Continue the template by appending the following text:

«this.title»

«this.description»

Obviously, this piece of template code will create part of the body of the HTML page. Again, we will access attributes of the current Form (as read from the model) and insert their values into the template (in this case, the title and the description). By the way, you can omit the this. prefix in front of the variable names.

The template goes on with the following text:

        «EXPAND field FOREACH this.fields»

      






«ENDFILE»
«ENDDEFINE»

The EXPAND statement will invoke yet another subtemplate with the name field. This sub template will be called for each element in the fields attribute (reference) of the current Form.

You might recall that we we defined three different kinds of >Fields in the metamodel:

1. Field (which is the super class for the other two field types
2. TextField
3. MultiLineTextField

As their names imply, a TextField will be a single line text entry field, whereas MultiLineTextField will be a multiline text input field. We somehow need to be able to render different HTMNL code for each of these different text field types.

As mentioned in the introduction, Xpand is not only type safe, but also supports polymorphic dispatch. This basically means we will create three templates (one for each of the different field types) with the same name. When evaluating the code template, the Xpand generator will dispatch to the appropriate template by matching the most concrete type of the current model element.

Add the following code to the template file:

«DEFINE field FOR Field»
«ERROR "should not happen"»
«ENDDEFINE»
«DEFINE field FOR TextField»
        «this.label»:

«ENDDEFINE»
«DEFINE field FOR MultiLineTextField»
        «this.label»:
        
«ENDDEFINE»

As you can see, all three templates have the same name. Only the type they are bound to differs. This is enough to let Xpand know which template to choose according to the type of the current model element. Let's suppose Xpand is iterating the model and the current model element is a TextField. Although Field is a direct super type of TextField, Xpand will not invoke the first template («DEFINE field FOR Field»), but the second template («DEFINE field FOR TextField»), as this is the most concrete match for the type of the model element.

Running the Code Generator

If you have followed the above steps, running the code generator is a piece of cake:

Open the context menu on src/workflow/workflow.mwe and select Run As -> MWE Workflow

This will start the code generator. You will see some log messages in the console view. If all went well, the output in the console reads something like this:

...
1503 INFO  Generator          - Written 1 files to outlet [default](src-gen)
1503 INFO  WorkflowRunner     - workflow completed in 650ms!

The result of the code generation can be found in src-gen/contact.html. As this file has some dependencies to CSS/image files, please download the files from the static folder (here) and place them in your project before opening contact.html in your browser.

Where to go from here

If you want to learn more about Xpand, be sure to attend my talk Use models and let the computer do the grunt work with Xpand at EclipseCon 2010. I'll show some more advanced topics in this talk (such as generator cartridges, using Xtend to augment your models, partitioning your code templates, using other metamodels to define your models).

Xpand comes with an extensive documentation (just go to Help -> Help Contents -> Xpand Documentation in Eclipse). You can also get help on Xpand in the Eclipse Community Forums

Should you need help, itemis (the company I work with) offers training and consulting for Xpand and a host of other modeling related technologies.

No doubt you have heard about Xtext. Xpand and Xtext go together great: you can use Xtext to define the structure of your models and create great-looking text editors to edit your models. Then, use Xpand to create code generators that take your textual models and turn them into running software. Actually, Xtext comes with a wizard that you to create a code generator project for your DSL.

Downwloads

The code for this tutorial can be found in my SVN repository on Google Code.

You guessed it, these are not the questions from a well-know Woody Allen movie, but questions that will be answered at the Eclipse Modeling Days which will be held in New York and Toronto in November.

So if you are wondering why everybody in Europe is so thrilled about Modeling, attend to one of these events - you'll get the answers there.

For your convenience, here is a list of some of the talks:

• Case Study: Model Transformations - Richard Woods, Credit Suisse
• Building DSLs with Xtext - Heiko Behrens, itemis
• Modeling on an enterprise scale - Eike Stepper, Eclipse Modeling Project
• Query, Transaction and Validation - Bernd Kolb & Boris Gruschko, both SAP
• Agent-Based Computer Simulation for Operational Risk Analysis - Ed MacKerrow, Los Alamos Labs
• Papyrus: Advent of an Open Source IME - Kenn Hussey, Zeligsoft
• B3: A Model for Unbreakable Builds
• There's a Modeling Solution for That - Bruce Trask, MDE Systems & Angel Roman, MDE Systems

If you ask me, this is a pretty decent line-up, both content-wise and people-wise. Not being one of the big conferences, the Modeling Days will actually give you the chance to talk to the experts and other end-users. Remember, networking is key.

So go ahead and register by adding your name to this wiki page. Registration is free, but you need to register in order to help the organizers to plan the catering. If you don't have an account for editing the wiki page or don't want to create one, just send mail to eclipse_modeling_nyc@eclipse.org or eclipse_modeling_toronto@eclipse.org

On Monday, Bernd, Kenn and I hosted the official Modeling BOF:

The topics discussed were focussing on consumability and long-term sustainability of EMF:

Documentation and Marketing

We all agreed that the entire modeling project lacks a decent documentation. While of course all of the projects and subprojects do have their own documentation, there is so such thing as an overview documentation. Modelers as we are, we quickly realized we're lacking a documentation superstructure. The idea is to provide a guide for users (newbies and seasoned experts alike) to navigate the Eclipse Modeling world. The modeling project offers a great variety of technologies, so it is not always easy to find out which set of technologies / frameworks you might want to use - especially if you are new to EMF.

We also agreed that we need more success stories (in the form of blog entries, articles and shiny flyers for the modeling landing page). A quick poll revealed that more than 90% of all people attending have been working on projects in which modeling was successfully applied, so there should be enough material to create success stories!

Kenn mentioned the Eclipse Foundation wants to start a documentation project (much like Babel) to improve overall documentation, we might eventually join this project. In the meantime, we decided to form a working group comprised of members of the Eclipse Modeling Project that starts to create this overview documentation. The initial members of this working group will be Kenn, Bernd, Peter, Tamer, Pierre, Marcello, Thibault and Anthony (guys, please send me your email addresses). The working group will start its work on the EMF dev mailing list. You are welcome to join us there.

Consuming Modeling Technology

I think there is no other technology that causes more emotions in an Eclipse committers life than P2. So we also had our little discussion about how to best provision Modeling technology. No big news here - we just re-iterated the pros and cons of using ZIPs vs. P2 and also had a short discussion about the Friends of Eclipse Download Wizard (which, rumor has it, will allow you to assemble a custom distro by just selecting the pieces you need). Several people mentioned that Amalgamation is intended to be a place for modeling distros.

EMF 3.0

We also discussed whether we should have a new major release of EMF that allows us to break API. Though not everybody agreed that we need a new major version (because the current version works smoothly for most people) we decided to set up an additional BOF to discuss this topic in more detail. I will write about the EMF 3 BOF later (yes, I am teasing)

Miscellanea

Someone asked if we need something like a model bus, the answer was no - we've got the modeling workflow engine (MWE).

We also briefly discussed the recent change of leadership on the MDT OCL component (Christian Damus revoked his committer and component lead status recently). Everybody agreed that to lead a component, a long-term committment is needed (which has been met by Christian in the most excellent way). We also agreed that we need to make avoid situations in which a whole bunch of projects / components suddenly are without a leader. This will be a task for the PMC.

After the official part of the BOF, there still was time to chat and shake hands with people you never met in person before. I had the pleasure to shake hands with Anthony Hunter, who works for IBM. From our discussions on mailing lists and bugs, you could guess that we have a totally different view on the marketing of software, but it turns out we're actually not that far apart

BOFs are a great way to interact with like-minded people at conferences. So, next time you go to a conference, make sure you participate in a BOF to make yourself be heard and take part in the discussion.

This week, we're going to have a look at how to update the data, add more records and how to query the database for information using HQL, the Hibernate Query Language.

If you haven't done so, you might consider taking last week's tutorial in order to get your environment set up and understand the basic concepts. If you're lazy, you can as well download the code of the tutorial to get started more quickly. It is available here.

As it turns out, in order to change anything in the library database, we must first fetch the to-be-changed data, so we need to have a look at querying first.

Retrieving and updating data
Let's assume we'd want to the update the number of pages in a book, because the new edition has an additional chapter.

1. So, first of all we need to open a session and begin a transaction:

       {
           Session session = sessionFactory.openSession();
           session.beginTransaction();

2. Next, let's create an HQL query. We want to find any books that are written by an author by the name "A. K. Dewdney" that contain "Omnibus" in their title.

           Query query = session.createQuery(
               "SELECT book from " +
               "    Book book, " +
               "    Writer writer " +
               "WHERE " +
               "    book.title like '%Turing Omnibus%' " +
               "AND " +
               "    writer.name = 'A. K. Dewdney'");

3. We can now execute the query and display the results:

           List books = query.list();
           Book book = books.get(0);
           System.out.println(book.getTitle());

4. Updating the page count is pretty obvious:

           book.setPages(520);

5. Finally, don't forget to commit the transaction and close the session:

           session.getTransaction().commit();
           session.close();
       }

Adding data
Let's now assume we want to add more books (and authors) to the library.

1. By now, you should be pretty familiar with the pattern of opening a new session:

       {
           Session session = sessionFactory.openSession();
           session.beginTransaction();

2. As we want to add new items to the library, we need to retrieve the library instance first of all:

           Query query = session.createQuery("from Library");
           List libraries = query.list();
           Library library = libraries.get(0);

3. Creating a new book and its author is easy, as we just have to use the API EMF so kindly generated for us:

           Writer writer = LibraryFactory.eINSTANCE.createWriter();
           writer.setName("J.R.R. Tolkien");
           Book book = LibraryFactory.eINSTANCE.createBook();
           book.setTitle("The Hobbit");
           book.setPages(320);
           book.setAuthor(writer);
           book.setCategory(BookCategory.MYSTERY);
           library.getBooks().add(book);
           library.getWriters().add(writer);

4. Finally, commit the transaction and close the session:

           session.getTransaction().commit();
           session.close();
       }

Well, it turns out EMF can help a long way to store data in a database. Here is how.
Prepare your development environment

1. Get Eclipse 3.5 M5 (click here to download)
2. Unpack and start Eclipse
3. Bring up the "Install New Software" dialog (Help -> Install New Software)
4. Select Teneo EMF Hibernate Runtime and Teneo EMF Hibernate SDK, version 1.0.3
5. Cick Finish
6. You most probably will be asked to restart Eclipse.

Create a target definition that includes Hibernate and HSQLDB
In order to keep things simple, we will store the data in an HSQLDB database. We will use the Hibernate OR Mapper to perform the mapping between your data objects and the database. As you might guess, quite a number of libraries will be involved to get the task accomplished. Instead of creating a bunch of plug-in projects containing the respective libraries, or - even worse - copying all libraries into our project, we'll set up a target definition. Target definitions help to maintain a common set of dependencies for all developers on a team, which is a Good Thing.

1. Create a new project library.target (File -> New -> Project... -> General -> Project)
2. Create three folders in this project: hibernate, dependencies, hsqldb
3. Go to the SpringSource Bundle Repository and download the following OSGi bundles:
   

   file	save in folder
   com.springsource.org.hibernate-3.2.6.ga.jar	library.target/hibernate
   com.springsource.org.apache.commons.logging-1.1.1.jar	library.target/dependencies
   com.springsource.org.dom4j-1.6.1.jar	library.target/dependencies
   com.springsource.org.apache.commons.collections-3.2.0.jar	library.target/dependencies
   com.springsource.javax.transaction-1.1.0.jar	library.target/dependencies
   com.springsource.antlr-2.7.7.jar	library.target/dependencies
   com.springsource.org.hsqldb-1.8.0.9.jar	library.target/hsqldb

4. Create a new target definition library.target in this project (File -> New -> Other... -> Plug-in Development -> Target Definition)
5. Open the target definition and add the three directories to it's contents. As the GUI does not allow you to work with relative paths, you might consider to use a text editor to paste the following text:

   <?xml version="1.0" encoding="UTF-8"?>
   <?pde version="3.5"?>
   <target description="Teneo-related stuff, mostly Hibernate" name="Library Target Definition">
       <locations>
           <location path="${eclipse_home}" type="Profile"/>
           <location path="${workspace_loc}/library.target/hibernate" type="Directory"/>
           <location path="${workspace_loc}/library.target/dependencies" type="Directory"/>
           <location path="${workspace_loc}/library.target/hsqldb" type="Directory"/>
       </locations>
   </target>

6. Open the target definition in the Target Definition Editor and click on the Set as Target Platform hyperlink in the upper right area. This will activate the target definition. All contained bundles are now available and can be referenced as dependencies.

Create a model for your data
The data model will be based on the well-known library tutorial that ships with EMF. If you are interested in a more in-depth look, I recommend taking this tutorial. However, to shortcut things, here is the ultra-slim version of the EMF-Tutorial:

1. Download the Rose class model and save it on your computer
2. Create a new EMF project (File -> New -> Other... -> Eclipse Modeling Framework -> EMF Project)
   • Project name: library
   • Model importer: Rose class model
   • Browse to the Rose class model library.mdl mentioned before
   • Click on Load to load the model
   • Click Next, then Finish
3. In the library.genmodel editor, right-click on the Library node and select Generate Model Code

Create the library main application
In order to demonstrate how to use the data model and how to perform CRUD operations with your data, we will create a simple Java class. In the good spirit of encapsulation and components, we will create a new plug-in project to host this class:

1. Create a new Plug-in project library.main (File -> New -> Project... -> Plug-in Project)
2. Open the manifest and add the following dependencies:
   • library (this is the bundle which contains our data model)
   • org.eclipse.emf.teneo.hibernate
   • org.eclipse.emf.ecore.xmi
   • com.springsource.org.hibernate
   • com.springsource.org.apache.commons.logging
   • com.springsource.org.dom4j
   • com.springsource.org.apache.commons.collections
   • com.springsource.javax.transaction
   • com.springsource.antlr
   • com.springsource.org.hsqldb
3. Create a new Hibernate configuration file hibernate.properties in library.main/src and paste the following lines:

   hibernate.connection.driver_class=org.hsqldb.jdbcDriver
   # use the following line to run the embedded db:
   # hibernate.connection.url=jdbc:hsqldb:file:/some/path/on/your/computer/dbname
   # the following line will connect to a standalone (local) DB server:
   hibernate.connection.url=jdbc:hsqldb:hsql://127.0.0.1/library
   hibernate.connection.username=sa
   hibernate.connection.password=
   hibernate.dialect=org.hibernate.dialect.HSQLDialect
   hibernate.hbm2ddl.auto=true

Implement the library main application
With all the boilerplate in place, we're finally ready to write some code. We will create a new class and add some code to create an author and his book and store both in a library.

1. Create a new class LibraryDemo, making sure it has a main method
2. In order to use Teneo to persist our data, we first need to create a datastore and register our model package with it:

       // create the data store
       String dataStoreName = "LibraryDataStore";
       HbDataStore dataStore = HbHelper.INSTANCE.createRegisterDataStore(dataStoreName);
   
       // register the model package with the data store
       dataStore.setEPackages(new EPackage[] { LibraryPackage.eINSTANCE });
   
       // initialize the data store, which creates the tables
       dataStore.initialize();

3. Next, we need to get hold of a session factory and request a new session form it:

       SessionFactory sessionFactory = dataStore.getSessionFactory();
       {
           Session session = sessionFactory.openSession();
           session.beginTransaction();

4. Now, let's create a new library and save it to the session:

           // create a library
           Library library = LibraryFactory.eINSTANCE.createLibrary();
           library.setName("Developer's bookshelf");
   
           // store the library
           session.save(library);

5. In the following part, we will create an author and his book, link them to each other and add them to the library. There is no specific Teneo aspect to this part of the code, it is just straightforward usage of the API EMF generated for your datamodel:

           // create an author
           Writer writer = LibraryFactory.eINSTANCE.createWriter();
           writer.setName("A. K. Dewdney");
   
           // create a book
           Book book = LibraryFactory.eINSTANCE.createBook();
           book.setTitle("The New Turing Omnibus");
           book.setPages(480);
           book.setCategory(BookCategory.MYSTERY); // oh well, let's hope it's not mystery to most readers!
           book.setAuthor(writer);
   
           // add book and writer to library
           library.getBooks().add(book);
           library.getWriters().add(writer);

6. Finally, we need to commit our changes to the database and close the session:

           // commit changes to the database and close the session
           session.getTransaction().commit();
           session.close();
       }

Start the DB server and run the application

1. Open a command line and navigate to the directory that contains hsqldb.jar
2. Start the HSQLDB server using this command line:

   java -cp com.springsource.org.hsqldb-1.8.0.9.jar org.hsqldb.Server -database.0 file:library -dbname.0 library

3. Finally (!) go back to Eclipse and start LibraryDemo. You should get an output similar to this one:

   Mar 6, 2009 1:50:26 PM org.eclipse.emf.teneo.hibernate.HbHelper createRegisterDataStore
   INFO: Creating emf data store and registering it under name: LibraryDataStore
   ...
   Mar 6, 2009 1:50:28 PM org.hibernate.tool.hbm2ddl.SchemaUpdate execute
   INFO: schema update complete

4. To actually see the data stored in the database, navigate to the directory containing the database and open library.log:

   CREATE USER SA PASSWORD "" ADMIN
   /*C1*/SET SCHEMA PUBLIC
   CONNECT USER SA
   ...
   INSERT INTO "library" VALUES(1,'Library',0,'Developer''s bookshelf')
   INSERT INTO "writer" VALUES(1,'Writer',0,'A. K. Dewdney',NULL,NULL,'Library','1',-2)
   INSERT INTO "book" VALUES(1,'Book',0,'The New Turing Omnibus',480,'Mystery',1,NULL,NULL,'Library','1',-3)
   DELETE FROM "writer" WHERE E_ID=1
   INSERT INTO "writer" VALUES(1,'Writer',0,'A. K. Dewdney',1,0,'Library','1',-2)
   DELETE FROM "book" WHERE E_ID=1
   INSERT INTO "book" VALUES(1,'Book',0,'The New Turing Omnibus',480,'Mystery',1,1,0,'Library','1',-3)
   INSERT INTO "writer_books" VALUES(1,1,0)
   COMMIT
   DISCONNECT

In the next installment, I will show you how to retrieve objects from the database and query the database using Hibernate Query Language (HQL).

Download the source
If you are interested in the source for the solution so far, you can download it here: (click to download).