Free Online Courses for Software Developers - MrBool
× Please, log in to give us a feedback. Click here to login
×

You must be logged to download. Click here to login

×

MrBool is totally free and you can help us to help the Developers Community around the world

Yes, I'd like to help the MrBool and the Developers Community before download

No, I'd like to download without make the donation

×

MrBool is totally free and you can help us to help the Developers Community around the world

Yes, I'd like to help the MrBool and the Developers Community before download

No, I'd like to download without make the donation

The MT4J framework – Multi-Touch Programming with Java – Part 3

Learn how to create applications that enable multi-touch interaction in Java

  The interactivity of the most traditional computer applications comes down to using a mouse and keyboard. The popularity of mobile devices, especially smartphones and tablets, increased the demand for applications where the interaction is performed directly on the place where the information is displayed.

The combination of gestures and elements of an interface that can be operated through these gestures is currently represented by the acronym NUI (Natural User Interfaces). Despite this new way of interacting with electronic devices to be increasingly present in everyday life, there are few professionals who know how to develop interfaces and applications that can extract the true potential of these interfaces.

  Based on this context, this article will present how to work with multi-touch interfaces in Java through the use of a framework. The aim of this article is initially present what are the applications, hardware, software and the possibilities that multi-touch interaction, combined with a suitable interface, can provide. The article also shows how to install, configure and work with the MT4J framework for the development of multi-touch platform-independent applications. Based on what was presented in this article the reader will have an overview of how to implement multiple interfaces that recognize gestures and thus adapt what was seen in their own projects.

  This third and last part of the article focus in the M4TJ framework and explain how to work with the object of this open-source Java based framework that relies on the TUIO protocol to create multi-touch applications

The MT4J framework

  Initially we will see how to configure the environment of the framework MT4J with the Eclipse development tool. To simulate the interaction of multiple touches without a specific hardware we will see how to employ more than a mouse connected to different USB ports on the same computer. This article is based on version 0.98 of the M4JT which is available free of charge via the file mt4jFull_098_prev.zip. This compressed file contains the source code and binary files of the MT4J framework, the libraries used, several examples, the documentation in JavaDoc format and additional files.

After downloading the file it is required to configure the Eclipse to be able to use and test the examples of the framework. The first step is to open Eclipse and click on File menu> Import as shown in Figure 1.


Figure 1
. Choosing the Import option from the Eclipse's File menu.

  Then you must choose the
Existing Projects into Workspace within the folder General of the window that appears after choosing the Import option and then click the Next button, as seen in Figure 2.



Figure 2. Choosing the option to import the project

  The next step is to type the file mt4jFull_098_prev.zip in the field Select archive file. After choosing the file the window automatically show that there are two projects compressed: mt4j-core and mt4j-desktop containing the source code of the MT4J framework and a project with examples and other files, respectively. Keep the two projects selected and click Finish to begin importing the project files into the current Eclipse workspace, according to Figure 3.



Figure 3. Selecting the projects of the MT4J framework.

  After the importing Eclipse automatically show in the
Package Explorer window the entire file structure of the imported projects. Figure 4 shows the hierarchy of folders for the project mt4j-desktop.



Figure 4. Hierarchy of folders and files of the project mt4j-desktop.

  To setup the framework MT4J so it accepted more than a mouse as the device it is necessary to change the text file Settings.txt (this setup works only on Windows). Before that make sure that at least two different mice are plugged into USB ports and that the movement of either one affects the same operating system cursor. The same idea can be applied to a mouse and a laptop’s track pad. Change the setting MultiMiceEnabled of the Settings.txt file so that it has the property MultiMiceEnabled = true. You can also set other options in this file such as the option of running applications that will fill the entire screen (Fullscreen = true).

  Now that the environment is setup let's see how to open the famous basic example known as "Hello World!" with the MT4J framework. This example is within the examples folder in the package basic.helloWorld and consists of two files: StartHelloWorld.java and HelloWorldScene.java. Just run the file StartHelloWorld.java in Eclipse as a Java application (Java Application) to obtain the result shown in Figure 5.




Figure 5
. Hello World! Example with the MT4J framework.

  The Hello World example in the MT4J framework basically creates a window and puts the phrase "Hello World!" in the middle of the screen. This phrase was inserted into a control that displays the text positioned in the middle of the window allowing the interaction with the user through all the gestures presented in Figure 1, that is, you can click, move and increase or decrease the size of the text. Every movement of mice in this interface generates a filled red circle and when there is a touch in the screen (left mouse click) on the text "Hello World!" the red circle is placed inside a blue circle to indicate that an action is being conducted over an interactive object. In Figure 5 the two mice with the left button pressed into the text show how to simulate the interactions of rotation and zoom movement without necessarily using a multi-touch sensitive surface.

  The MT4J framework contains several examples including how to use controls similar to elements that exists in traditional applications (such as buttons, text boxes, rectangles, lines, etc), integration with OpenGL technology and the SVG file format, visualization of 3D models (created in applications such as Maya or Blender), physics engine games with 2D, 2.5D and 3D, manipulation of maps, public access to APIs such as Flickr and other examples. The page http://www.mt4j.org/mediawiki/index.php/Examples contains several videos showing the execution of some of these examples in a multi-touch table. Let's see how the example "Hello World!" was created to understand the basics of using this framework.

  The example is divided into two classes: StartHelloWorld and HelloWorldScene. The commented source code of the StartHelloWorld class is shown in Listing 1.

Listing 1. Source of StartHelloWorld class containing the example Hello World! in MT4J.
package basic.helloWorld;

import org.mt4j.MTApplication;

// You must inherit from the class MTApplication to create applications in MT4J

public class StartHelloWorld extends MTApplication

{

  // Entry point of application: calls the initialize() method of the MTApplication class

  // This method will call the startUp() method

  public static void main(String[] args)

  {

    // This method makes the necessary initialization of the framework

    // such as reading the file Settings.txt

    initialize();

  }

  @Override

  public void startUp()

  {

     // Here a scene is added to the application.

     // Note that the object of the StartHelloWorld class is sent

     // in the first parameter of the constructor of the scene (this object)

     // This first scene automatically becomes the current scene and receives

     // touch events

    addScene(new HelloWorldScene(this, "Hello World Scene"));

  }

}

  To use the framework MT4J one must first create a class that inherits from MTApplication. In the example the class StartHelloWorld inherits from MTApplication and inside its main() method a call is made to the initialize() method of the class MTApplication that, among other things, setup the environment, reads the file Settings.txt and also makes a call to the startUp() method. The startUp() method is responsible for creating a scene (scene), a fundamental concept of the MT4J framework.

  A scene in MT4J can be viewed as a work area that will contain other elements. This scene will have a graph with all the controls and also a canvas object that will assume the role of container for the objects. Note that a scene is not responsible for controlling the receiving of messages of the TUIO protocol as this is done transparently by the class that inherited from MTApplication.

  The StartHelloWorld's startUp() method calls the method addScene() which takes as a parameter an instance of the class HelloWorldScene and a string with the name of the scene. The code of the HelloWorldScene class is shown in Listing 2.

Listing 2. Content HelloWorldScene class which contains a simple scene with an interactive control MTTextArea.

package basic.helloWorld;

import org.mt4j.AbstractMTApplication;

import org.mt4j.components.visibleComponents.widgets.MTTextArea;

import org.mt4j.input.inputProcessors.globalProcessors.CursorTracer;

import org.mt4j.sceneManagement.AbstractScene;

import org.mt4j.util.MTColor;

import org.mt4j.util.font.FontManager;

import org.mt4j.util.font.IFont;

import org.mt4j.util.math.Vector3D;

// One must inherit the class AbstractScene to create a new scene

public class HelloWorldScene extends AbstractScene

{

  // The constructor takes the object AbstractMTApplication and the name of the scene

  public HelloWorldScene(AbstractMTApplication mtApplication, String name)

  {

    // Calling the constructor of AbstractScene for initializations

    super(mtApplication, name);

                

    // Clear the background of the scene

    MTColor white = new MTColor(255,255,255);

    this.setClearColor(new MTColor(146, 150, 188, 255));

    // Here we indicate that this scene will receive the input events (the touches)

    // Basically we create a object CursorTracer that represents the class

    // that get all the touch events

    this.registerGlobalInputProcessor(new CursorTracer(mtApplication, this));                

                

    // Creating the font that will be used to display the text

    IFont fontArial = FontManager.getInstance().createFont(mtApplication, "arial.ttf",

      50,         //Font size

      white);        //Font color

    // Create an object MTTextArea containing the interactive text

    MTTextArea textField = new MTTextArea(mtApplication, fontArial);

                

    // Set visual text settings

    textField.setNoStroke(true);

    textField.setNoFill(true);

                

    // Inserting the content to be shown in the control

    textField.setText("Hello World!");

    // Centering the text on the screen

    // Note that the coordinates are obtained from the object AbstractMTApplication

    // that was received as a parameter

    textField.setPositionGlobal(

    new Vector3D(mtApplication.width/2f, mtApplication.height/2f));

    // Adding text to the scene's canvas. Place the object MTTextArea on the canvas

    //  make the connection of events with their basic functionality (

    // move, rotate, increase or decrease the text size)

    this.getCanvas().addChild(textField);

  }

  // The method OnEnter() is called when the application focus changes to this scene

  public void onEnter() {}

        

  // The method OnLeave() is called when this scene comes is out of the application's focus

  public void onLeave() {}

}

  The HelloWorldScene class inherits from the abstract class AbstractScene because it represents a scene from the framework. The constructor of this class should receive an object AbstractMTApplication and also a String containing the name of the scene. Early in the constructor method we must call the constructor for the AbstractScene in order to initialize the scene. From this moment we can include user interface controls in the scene.

  The constructor method of HelloWorldScene creates an object MTColor and then calls the method setClearColor() to clear the screen. Incidentally, almost all controls and visual elements of this framework starts with the prefix MT. Then we have the method call registerGlobalInputProcessor() from AbstractScene. This call is very important because through it we indicated to the object mtApplication (received as the first parameter of the class constructor HelloWorldScene) that the AbstractScene object should receive the events that will be forwarded to each of the controls of this scene. To register and receive events is necessary to pass an object of the CursorTracer class as a parameter to the method registerGlobalInputProcessor(). After this call the touch events received through the TUIO protocol will be forwarded to any interactive control that was inserted into the canvas object of this scene in a transparent manner.

  The method continues creating an object using a specific font and the only interactive element of this scene: an object of class MTTextArea called textField. This object receives as a parameter in its constructor not the specific scene but the object mtApplication. Then two visual options are configured along with its content and positioning in the scene. Finally, the textField object is added to the scene's canvas by calling the addChild(), which belongs to the canvas object returned by the getCanvas() of the AbstractScene class. Finally, the methods OnEnter() and OnLeave() are inserted in the class HelloWorldScene without content. The goal of these methods is to perform some necessary treatment when the scene is activated or when the scene is turned off, respectively. In the framework MT4J you can work with more than one scene per application and switch between them, but only one scene is active and receives touch events at a time.

  Once the control MTTextArea was inserted into the canvas and the scene has already been configured to receive events through an object of the class CursorTracer all touch events that occur in the area that is part of MTTextArea are routed to this control. The MTTextArea as well as various other controls provided by this framework is capable of performing the basic interactions of movement, rotation, size increase and decrease automatically without the need of inserting code. However if the programmer desires he/she can create customized controls that dealt manually when a touch is generated within the coordinates of the control area, when a touch is removed from the control's area or when a touch moves within the coordinates of the control's area.

Conclusion

  The interaction with an application through gestures combined with a suitable interface for this interaction has been increasingly exploited in various devices such as smartphones, tablets, kiosks, tables and other multi-touch devices. Due to the demand for applications that have this type of interaction, this article focuses on developing applications that use multi-touch interfaces in Java using the framework MT4J. Initially the article shown what are the applications, hardware, and software applications that multi-touch interaction can provide. The article also describes how to install, configure and work with the MT4J framework for the development of multi-touch platform-independent applications.

  This last part focused on the MT4J framework and its details such as the objects and concepts required to work with this Java framework.


To see the first part, access: http://mrbool.com/p/Hardware-and-software-for-multi-touch-applications-Multi-Touch-Programming-with-Java-Part-2/23183 

References                                                                                                                                   

http://www.mt4j.org/mediawiki/index.php/Main_Page

Framework MT4J.

http://www.tuio.org/

Specification and implementations of the protocol TUIO..

http://www.hp.com/united-states/campaigns/touchsmart/

Information on the TouchSmart line of computers from HP.

http://www.lukew.com/ff/entry.asp?1071

Touch Gesture Reference Guide.

http://research.microsoft.com/en-us/um/people/benko/projects/sphere/

Information about the Microsoft Sphere.

http://www.gesture-cube.com/

Information about the Gesture Cube.

http://www.microsoft.com/hardware/en-us/products/touch-mouse/microsite/

Information about the Microsoft Touch Mouse.

http://hci.rwth-aachen.de/benddesk

Information about the BendDesk.

http://www.hpi.uni-potsdam.de/baudisch/projects/multitoe.html

Information about the multi-touch floor.

http://www.pranavmistry.com/projects/sixthsense/

Information about the SixthSense project.

http://www.instructables.com/id/Interactive-Multitouch-Display/

A guide for creating multi-touch table with low cost.

http://www.microsoft.com/surface/en/us/default.aspx

Information about the Microsoft Surface.



Mauro Pichiliani has the Master of Science degree on collaborative systems by the Aeronatics Institute of Technology (ITA) in Brazil. He is a specialist on database technologies with more than 8 years of experience on the industry...

What did you think of this post?
Services
[Close]
To have full access to this post (or download the associated files) you must have MrBool Credits.

  See the prices for this post in Mr.Bool Credits System below:

Individually – in this case the price for this post is US$ 0,00 (Buy it now)
in this case you will buy only this video by paying the full price with no discount.

Package of 10 credits - in this case the price for this post is US$ 0,00
This subscription is ideal if you want to download few videos. In this plan you will receive a discount of 50% in each video. Subscribe for this package!

Package of 50 credits – in this case the price for this post is US$ 0,00
This subscription is ideal if you want to download several videos. In this plan you will receive a discount of 83% in each video. Subscribe for this package!


> More info about MrBool Credits
[Close]
You must be logged to download.

Click here to login