# WebHookIt first release - developer preview

I’m very pleased to announce the birth of WebHookIt !

Visual scripting for the web, Pipes for hackers, using Node.js, and mongoDB
WebHookIt is kind of an open-source Yahoo! Pipes, but with a wider scope of applications. The idea is to have a very HTTP-centric visual language to write glue-code between web-services using Node.js.

It’s made to be customized/modified/enhanced :

• easily add core modules (written in Node.js)
• customize UI (forms and modules, using WireIt and inputEx)
• customize the execution engine
• composability, reuse modules
It’s perfect for :

• Notifications (email, sms, IM, social networks)

• services integration and communication
• plugins for web applications (Webhooks)
• chaining web-services
• providing simple webservices
It’s not a hosted service like Yahoo! Pipes, but you can install it on your local machine, your servers, on Joyent Node.js SmartMachines, and maybe even on your Palm Pre devices (lucky you!)

We are not there yet

This first version of WebHookIt is still alpha software. APIs may change and some basic features are still missing. Those are going to be adressed in the first stable release :

Here is a video of WebHookIt in action against the Prowl API : (sorry for the music… it won’t happen again !)

# YUI Builder: to use or not to use ? Use yproject !

If one of your project is using the YUI library, you should definitely consider using the YUI Builder. For YUI3, I would go event further in saying that it is mandatory. Here is why.

YUI Builder contains the component build tool, which is the ANT based build system YUI uses to generate the code in the YUI 2 and YUI 3 build directories, from the contents of the src directories for each component.
Ok, looks cool to me. But what this description doesn’t mention, is that you can use the Builder for your projects too ! The documentation is precise and concise. The YUI Builder has a lot of nice features, however, it comes at a price :

Drawbacks

• requires a bigger stack : you’ll need java, ANT, and YUI Builder to be installed. While most developers should have those tools already installed, open-source projects may suffer contribution-loss for having such a deep stack to install, understand and configure. Hopefully, for existing components, a simple _ant all _will work.

• build step : For each modification, you’ll have to re-build the component. I strongly recommend to have some kind of IDE integration. Obviously, ANT and Java were chosen for a good reason: auto-building in Eclipse is a breeze. If you are a Textmate user, here is a simple command to launch ANT.

• build files to create when adding a new module : build.properties, build.xml, create the directory structure, setup the test files, etc… But this is kind of a stupid task: some tool could help with that ! (read on !)

• Structure : Your modules will have the same structure as YUI components. Those conventions will help you organize your files, ensuring homogeinity through your project. This should also make it easier to understand for existing YUI users.

• JSlinting every module : you’ll be surprise how many errors you can catch before trying your code in the browser… Although your IDE probably already runs JSlint on each save, it is good to know that this step is mandatory in your workflow even for notepad addicts.

• Minification : YUI compressor is built-in. You don’t need to even think about it.

• Module dependencies : When I started porting my librairies to YUI3 without using the YUI Builder, I spent quite some time setting up the dependencies of each module, because I needed to update it twice: in the module declaration and in the loader seed file. Using the YUI Builder removes this redundancy by declaring the dependencies only once in the build.properties file. The loader seed file can then be automatically generated. (more about this in the yproject paragraph.)

• Deploying to the YUI gallery (optional): The YUI gallery is the place to make your YUI contributions public, leveraging the Yahoo! CDN. All components in the gallery are automatically built with the YUI Builder.
yproject - With a Little Help from My Friends

yproject is a command-line utility to scaffold projects and components using the YUI3 library and the YUI Builder. It is open-source, and hosted on github. It tries to fill some gaps left by the YUI Builder.

It contains a project template and a module template to quickly bootstrap your project enforcing best practices from the start :

• loader module for dependencies : the YUI library loader component is automatically generated using a script called meta_join.py. yproject provides a similar script : read the build.properties files, and generate a myproject-loader seed file which will dynamically load all your dependencies.

• testing with YUI test and Selenium : the yproject template provides scripts to generate the tests.xml file and launch them all with Selenium.

• test coverage : the tests will by default run the instrumented code, and generate a code coverage report at the end of the Selenium batch run.

• documentation : the created project already contains a script to launch YUI Doc, and generate the API documentation into the api/ folder
yproject requires node.js. You can install using npm : npm install yproject

How to organize components

A last question remains: how should I organize my components ? At first glance, it seems to be a compromise :

• on one side, you can bundle a lot of functionality into one file to improve performance, limiting HTTP requests.
• on the other size, smaller components are easier to test and re-use.
However, YUI has two features, which make this decision a no-brainer :

• Combo-loading : Available from the Yahoo! CDN or using YUI PHP loader. This will combine multiple files into one HTTP request and cache it, making the next request insanely fast. I hope we will soon get a Rails plugin and a Node.js package to do the same…

• Rollup files : If you often use a set of component together, you can configure rollup components to build them all into a bigger component, limiting the HTTP requests.
So my final advice is the following :

Do as many components as you can ! Split them all into smaller pieces !

This will ensure reusability, easier testing, easier debugging etc… In short: true engineering practices.

Conclusion

YUI3 + YUI Builder is definitely harder to get started with, than any other Javascript library framework. But none can compare to the professional development workflow it provides. If you are targeting high quality standards, performance, well-documented projects, test-driven development and reusability, it really worth the trouble.

Be professional: use YUI Builder and yproject !

# inputEx 0.7.0 release and Roadmap

inputEx 0.7.0 was released today !

Most importantly, we upgraded YUI dependency to version 2.8.2r1 to fix security issues, and fixed a major bug for Internet Explorer 9.

We also worked on setting correct names for all sub-fields in order to use classical browser form posting. You can now safely use Group and List fields using standard forms.

Finally, I merged all the various Wiki pages we had into the new inputEx Guide. Obviously far from being complete, this first version will help us replying to questions from the forum, and hopefully help newcomers.

You can take a look at the complete changeset here, and download the 0.7.0 version here.

Now, where are we going ?

With the upcoming YUI 3.3.0 release, it is now time to seriously work on upgrading inputEx to YUI3.

Of course, this is not a new idea, and we’ve already tried to start the project over, leveraging the YUI 3 widget infrastructure. However, this appeared to be a huge amount of work, while throwing away most of the current code.

Facing our failure, we’ve decided to take a smaller step to get there, so we’ll start by forking the current version to make it work with YUI3, using yui2in3 for widgets dependencies, while keeping code changes to the strict minimum.

Once this version is out, we’ll be able to continue with the following goals in mind :

• remove unrelated code from the library (yql-related stuff, JsonTreeInspector, …)
• removing YUI2 dependencies while widgets get integrated into YUI3
• make inputEx available on the YUI gallery (and leverage Yahoo! CDN )
• finally use the YUI Widget framework
You will be able to follow the progress of the inputEx-yui3 branch on https://github.com/neyric/inputex/network.

Happy new year !

# Using Salesforce from Bonita Open Solution

Bonita Open Solution is an open-source Business Process Management ( BPM ). It comes with one of the best BPMN editor I have tried (Bonita Studio). The main feature of Bonita, is its extensibility, which is achieved by using either :

• Connectors: java classes which defined the task to run, as well as the interface to configure the connector from within Bonita Studio
• Scripts: Using Groovy scripts or Javascript through the Rhino connector
In this example, I will show how to use a Groovy script to access Salesforce query interface, but the process described here is pretty much the same for all Java librairies you may want to use from Bonita.

First, we need to generate the JAR file for your Salesforce organization. We’ll use the Force.com Web Service Connector (WSC), which generates the JAR file from your Enterprise WSDL:

• To get your WSDL, log into your organization and go to Your Name -> Setup -> App Setup -> Develop -> API. Generate the enterprise WSDL (unique to your organization) and save it as salesforce-mycompany.wsdl
• Open a console, and run the WSC tool :
java -classpath wsc-20.jar com.sforce.ws.tools.wsdlc \   salesforce-mycompany.wsdl salesforce-mycompany.jar

Now, open Bonita Studio and import the two JARs (Extensions -> Add/remove)

Then, we’ll design a simple process. The first task ‘Query on Salesforce’ will perform the SOQL query on Salesforce, and format the results into human-readable HTML. The second task ‘View results’ will just present the results to the user into a form.

In the first task, add a Groovy Script Connector and enter the following script :

Save the result of the script into a new data variable called ${result}. Finally, add a form to the second task. Insert a Text field, and sets its default value to the${result} variable created above.

Run the process and voilà !

I’ve also been using the same scripting capabilities to post articles to my blog using XML-RPC through the wordpress-java library and it worked like a charm !

Which libraries are you gonna use ?

# Hello world !

All of my french articles have moved to http://fr.neyric.com. It let’s you choose if you want to follow only one of the feeds. Make sure you update your feed reader !

Tous mes articles français ont été déplacés sur http://fr.neyric.com. Cela vous laisse choisir les flux que vous souhaitez lire. Mettez à jour vos lecteur de flux !

# WireIt 0.5.0

Hi everyone !

This release had 2 major goals:

• Enhance the WiringEditor
• Documentation documentation documentation
The WiringEditor received a lot of attention: by providing a full-page
editor, people have been able to start prototypes very quickly for
their projects. However, we needed to improve the way we connect this
editor to the backend, and that’s what we’ve done, through “adapters”.

I’m even very proud of this idea because… it’s not mine ! Big thanks to this insightful comment from Jonathan Lister and hurray for open-source communities!

documentation and many examples, it was still missing some guidelines.
I therefore wrote the WireIt guide: http://javascript.neyric.com/wireit/guide.html

Of course, we also have some minor improvements :

http://javascript.neyric.com/wireit/VERSION.txt

Have fun !
Eric

# WireIt 0.4.0 released

The long awaited release of WireIt is finally here ! Awaited indeed, since a major bug was still present in version 0.3.0, preventing it to work on IE. It has been fixed for a long time on the repository but has discouraged many people using WireIt (hopefully not too much…).

What’s new in version 0.4.0 ?

• inputEx integration: inputEx is another of my YUI libraries, to create fields and forms. Through a new class called “FormContainer”, you can now build WireIt containers using the advanced forms options from inputEx. The value is saved at the layer level for you through Layer.getWiring

• WiringEditor: This is my favorite feature. While still largely incomplete, the Wiring Editor provide a full-blown interface to create visual languages. It takes care of most of the interface you might want to build a visual editor on top of WireIt. The language definitions are written in JSON, I hope this will greatly reduce the learning curve of WireIt. The transparent ajax communication with the database is handled through a SMD webservice using YUI-RPC. WireIt provides a simple php backend that stores the JSON wiring states in MySql (used for the demos on wireit’s website).

• new options:

• terminals position in containers: {top: , left: , right: , bottom: } instead of [x,y]
• Terminal.alwaysSrc : force this terminal to be the source element in every connected wires
• Container.preventSelfWiring : prevent connections between two terminals of this container
• Four Wire methods for handling mouse events. You can override them for a custom use :

• API Documentation with YUI Doc (much more precise on properties and events)

• Layer Map is now clickable and draggable. It will move the linked layer accordingly.

• IE Bug fix ! (finally)

• Moved to github for git repository and better wiki

Build visual languages applications !

It’s now pretty easy to build visual languages applications. (or at least, prototype them…).

You can follow some projects on http://github.com/neyric/wireit/network. I’m especially looking forward to LeifW XProc Visual Editor

Here is a little presentation that I published on WireIt’s homepage :

# inputEx 0.2.2, SMD and YUI-RPC

0.2.2 release

Although we didn’t add any new field in this release, inputEx 0.2.2 still brings a new set of options and bugfixes. (see the changeset for details)

The most significant part of this version, which I’m pretty excited about, resides in the following components :

Abstract from the proposal:

A Service Mapping Description (SMD) is a JSON representation describing web services. An SMD can defines the various aspects of a web service such that clients can coherently interact with the web services. An SMD can be used by generic tools to generate interfaces, human and programmatic, to access available web services. A wide array of web services can be described with SMD including REST services and JSON-RPC services. The SMD format is designed to be flexible, compact, simple, readable, and easily implemented.
SMD is basically a WSDL-like, but for JSON. It’s so much simpler to implement that I regret the time I spent on WSDL ! Instead of using XML, XML-RPC and SOAP, we are now able to play with technologies better suited for the browser environment: JSON, JSON-RPC, REST, JSONP etc…

Concerning the “programmatic interface”, the dojo framework already provides a remote procedure calls module. However, since I’m a YUI-user, I had to re-implement it on YUI, which gave birth to YUI-RPC.

How inputEx is playing with SMD

From the inputEx point of view, the best part of SMD is that the method parameters are defined using… JSON-Schema !

inputEx was already able to build a form from a JSON-Schema, so it was pretty trivial to implement an automated form generation to call webservices based on a SMD. Here is an example of such a form using a SMD for Yahoo! search.

I believe this will help a lot in decoupling interfaces and webservices. To prove my point, I quickly wrote some SMDs for popular webservices like Yahoo! search, pipes, Twitter, geonames, delicious, flickr, YQL and others, and built a tiny interface, the SMD-tester, to build the forms, call those webservices, and display the response.

Have fun !

# JSON Schema and inputEx

Proposed by Kris Zyp, JSON Schema takes the good ideas from XML Schema. From http://json-schema.org/ :

JSON Schema is a specification for a JSON-based format for defining the structure of JSON data. JSON Schema provides a contract for what JSON data is required for a given application and how it can be modified, much like what XML Schema provides for XML. JSON Schema is intended to provide validation, documentation, and interaction control of JSON data.
There are numerous similarities between inputEx and JSON Schema. If you are not convinced, here is an example of a JSON schema:
{“description”:”A person”,

“type”:”object”,

 “properties”: {

“name”: {“type”:”string”},

“born” : {

 “type”:[“integer”,”string”],

“minimum”:1900,_

_ “maximum”:2010,

 “format”:”date-time”, _

_ “optional”:true

 },

“gender” : {“type”:”string”,

“options”:[

{"value:"male","label":"Guy"},

 {“value”:”female”,”label”:”Gal”}]},

“address” : {“type”:”object”,

“properties”:{

 “street”:{“type”:”string”},

“city”:{“type”:”string”},

“state”:{“type”:”string”}

}

}

 }

}
inputEx users should be comfortable writing a form for this schema :
{

type: “group”,

fields: [

{type: “string”, inputParams: {name: “name”, required: true}},

{type: “date”, inputParams: {name: “born”} },

{type: “select”, inputParams: {name: “gender”, selectOptions:[“Guy”, “Gal”], selectValues: [“male”, “female”]}},

{type: “group”, inputParams: {

fields: [

{type: “string”, inputParams: {name: “street”}},

{type: “string”, inputParams: {name: “city”}},

{type: “string”, inputParams: {name: “state”}}

] } } ] }
The main difference is that the JSON schema ontology is data-centric, wheras the inputEx JSON schema ontology is form-centric, but as you can see, they are very similar.

Another similarity is the self-descriptive ability of JSON Schema and inputEx. JSON Schema has its own schema, a schema for itself. inputEx has the same kind of self reference: the TypeField contains forms to describe the options of each field (you can see this at the end of TypeField.js). This self-referencing property has very nice implications: it gives the opportunity to create a form to build forms, and that’s exactly what the inputEx Builder is: a form which outputs inputEx JSON.

inputEx implementation of JSON Schema

In version 0.2.0, I added a (very) light implementation of a JSON Schema to inputEx JSON. Hopefully, Brad Clements saw the advantages of having automatic form generation from JSON schemas and took the prototype one (big) step forward by implementing JSON referencing. We are now able to create atomic schemas and combine them in nested schemas.

At the current state, inputEx doesn’t implement all the validation that can be described in JSON Schema, but we are on the way. On the other hand, inputEx has many options concerning the form interface, that are not necessary in JSON Schema, which focuses on data validation. Brad suggested an extension to JSON Schema by adding an “_inputex” property to type definition. It allows to set the additionnal interface options for inputEx within the schema.

Both of these features are demonstrated in this example which is built from this schema (JSON schema with inputEx extension).

The last feature that we’re missing is the extending capability (“extends”:{“\$ref”:”person”}), but the implementation should be pretty straightforward.

Round-trip: defining JSON-Schema through forms, and **Persevere backend**

Up to now, we focused on converting JSON schemas to inputEx forms, but the other way is perfectly conceivable ! Leveraging the inputEx Builder, we would have a nice interface to build JSON schemas and associated forms **at the same time. Combined **with the Persevere framework backend, we could provide a very nice administration interface, that let you edit models through the inputEx Builder, then edit the data through the generated inputEx forms. You could argue that some fields should not be visible in forms, but if you take a closer look at inputEx fields, two of them are perfect for this case: the HiddenField, which is not visible at all, or the UneditableField which lets you use a custom rendering method for an uneditable field.

I think this might end up in a more effective interface than the current Persevere database explorer because it creates the form interfaces at the same time, and make them reusable within your application.

A last idea, we could provide json-schemas for defining method parameters. inputEx could then automatically generate forms to call these methods (through JSON-RPC or locally).

Of course, if you’re interested in the following developments, we urge you to contact us through the forum.

Have fun !