I’m very pleased to announce the birth of WebHookIt !

Visual scripting for the web, Pipes for hackers, using Node.js, and mongoDB

WebHookIt is kind of an open-source Yahoo! Pipes, but with a wider scope of applications. The idea is to have a very HTTP-centric visual language to write glue-code between web-services using Node.js.

It’s made to be customized/modified/enhanced :

• easily add core modules (written in Node.js)
• customize UI (forms and modules, using WireIt and inputEx)
• customize the execution engine
• composability, reuse modules

It’s perfect for :

• Notifications (email, sms, IM, social networks)
• services integration and communication
• plugins for web applications (Webhooks)
• chaining web-services
• providing simple webservices

It’s not a hosted service like Yahoo! Pipes, but you can install it on your local machine, your servers, on Joyent Node.js SmartMachines, and maybe even on your Palm Pre devices (lucky you!)

We are not there yet

This first version of WebHookIt is still alpha software. APIs may change and some basic features are still missing. Those are going to be adressed in the first stable release :

• loop control
• conditional execution
• Community, pipes sharing, leverage the pre-existing package manager npm

All the links

• The online demo : http://webhookit.no.de (hosted @Joyent)
• Source obviously hosted on GitHub
• The documentation uses the awesome GitHub pages
• Forum here
• Report any Issue on GitHub issues

Here is a video of WebHookIt in action against the Prowl API : (sorry for the music… it won’t happen again !)

If one of your project is using the YUI library, you should definitely consider using the YUI Builder. For YUI3, I would go event further in saying that it is mandatory. Here is why.

YUI Builder contains the component build tool, which is the ANT based build system YUI uses to generate the code in the YUI 2 and YUI 3 build directories, from the contents of the src directories for each component.

Ok, looks cool to me. But what this description doesn’t mention, is that you can use the Builder for your projects too ! The documentation is precise and concise. The YUI Builder has a lot of nice features, however, it comes at a price :

Drawbacks

• requires a bigger stack : you’ll need java, ANT, and YUI Builder to be installed. While most developers should have those tools already installed, open-source projects may suffer contribution-loss for having such a deep stack to install, understand and configure. Hopefully, for existing components, a simple ant all will work.

• build step : For each modification, you’ll have to re-build the component. I strongly recommend to have some kind of IDE integration. Obviously, ANT and Java were chosen for a good reason: auto-building in Eclipse is a breeze. If you are a Textmate user, here is a simple command to launch ANT.

• build files to create when adding a new module : build.properties, build.xml, create the directory structure, setup the test files, etc… But this is kind of a stupid task: some tool could help with that ! (read on !)

Advantages

• Structure : Your modules will have the same structure as YUI components. Those conventions will help you organize your files, ensuring homogeinity through your project. This should also make it easier to understand for existing YUI users.

• JSlinting every module : you’ll be surprise how many errors you can catch before trying your code in the browser… Although your IDE probably already runs JSlint on each save, it is good to know that this step is mandatory in your workflow even for notepad addicts.

• Minification : YUI compressor is built-in. You don’t need to even think about it.

• Module dependencies : When I started porting my librairies to YUI3 without using the YUI Builder, I spent quite some time setting up the dependencies of each module, because I needed to update it twice: in the module declaration and in the loader seed file. Using the YUI Builder removes this redundancy by declaring the dependencies only once in the build.properties file. The loader seed file can then be automatically generated. (more about this in the yproject paragraph.)

• Deploying to the YUI gallery (optional): The YUI gallery is the place to make your YUI contributions public, leveraging the Yahoo! CDN. All components in the gallery are automatically built with the YUI Builder.

yproject - With a Little Help from My Friends

yproject is a command-line utility to scaffold projects and components using the YUI3 library and the YUI Builder. It is open-source, and hosted on github. It tries to fill some gaps left by the YUI Builder.

It contains a project template and a module template to quickly bootstrap your project enforcing best practices from the start :

• loader module for dependencies : the YUI library loader component is automatically generated using a script called meta_join.py. yproject provides a similar script : read the build.properties files, and generate a myproject-loader seed file which will dynamically load all your dependencies.

• testing with YUI test and Selenium : the yproject template provides scripts to generate the tests.xml file and launch them all with Selenium.

• test coverage : the tests will by default run the instrumented code, and generate a code coverage report at the end of the Selenium batch run.

• documentation : the created project already contains a script to launch YUI Doc, and generate the API documentation into the api/ folder

yproject requires node.js. You can install using npm : npm install yproject

Read more on https://github.com/neyric/yproject

How to organize components

A last question remains: how should I organize my components ? At first glance, it seems to be a compromise :

• on one side, you can bundle a lot of functionality into one file to improve performance, limiting HTTP requests.
• on the other size, smaller components are easier to test and re-use.

However, YUI has two features, which make this decision a no-brainer :

• Combo-loading : Available from the Yahoo! CDN or using YUI PHP loader. This will combine multiple files into one HTTP request and cache it, making the next request insanely fast. I hope we will soon get a Rails plugin and a Node.js package to do the same…
• Rollup files : If you often use a set of component together, you can configure rollup components to build them all into a bigger component, limiting the HTTP requests.

So my final advice is the following :

Do as many components as you can ! Split them all into smaller pieces !

This will ensure reusability, easier testing, easier debugging etc… In short: true engineering practices.

Conclusion

YUI3 + YUI Builder is definitely harder to get started with, than any other Javascript library framework. But none can compare to the professional development workflow it provides. If you are targeting high quality standards, performance, well-documented projects, test-driven development and reusability, it really worth the trouble.

Be professional: use YUI Builder and yproject !

inputEx 0.7.0 was released today !

Most importantly, we upgraded YUI dependency to version 2.8.2r1 to fix security issues, and fixed a major bug for Internet Explorer 9.

We also worked on setting correct names for all sub-fields in order to use classical browser form posting. You can now safely use Group and List fields using standard forms.

Finally, I merged all the various Wiki pages we had into the new inputEx Guide. Obviously far from being complete, this first version will help us replying to questions from the forum, and hopefully help newcomers.

You can take a look at the complete changeset here, and download the 0.7.0 version here.

inputEx Roadmap

Now, where are we going ?

With the upcoming YUI 3.3.0 release, it is now time to seriously work on upgrading inputEx to YUI3.

Of course, this is not a new idea, and we’ve already tried to start the project over, leveraging the YUI 3 widget infrastructure. However, this appeared to be a huge amount of work, while throwing away most of the current code.

Facing our failure, we’ve decided to take a smaller step to get there, so we’ll start by forking the current version to make it work with YUI3, using yui2in3 for widgets dependencies, while keeping code changes to the strict minimum.

Once this version is out, we’ll be able to continue with the following goals in mind :

• remove unrelated code from the library (yql-related stuff, JsonTreeInspector, …)
• removing YUI2 dependencies while widgets get integrated into YUI3
• make inputEx available on the YUI gallery (and leverage Yahoo! CDN )
• finally use the YUI Widget framework

You will be able to follow the progress of the inputEx-yui3 branch on https://github.com/neyric/inputex/network.

Happy new year !

Hi everyone !

I’m pleased to announce the latest version of WireIt, namely 0.5.0.

This release had 2 major goals:

• Enhance the WiringEditor
• Documentation documentation documentation

The WiringEditor received a lot of attention: by providing a full-page
editor, people have been able to start prototypes very quickly for
their projects. However, we needed to improve the way we connect this
editor to the backend, and that’s what we’ve done, through “adapters”.

I’m even very proud of this idea because… it’s not mine ! Big thanks to this insightful comment from Jonathan Lister and hurray for open-source communities!

Concerning the documentation, although we already had a full API
documentation and many examples, it was still missing some guidelines.
I therefore wrote the WireIt guide: http://javascript.neyric.com/wireit/guide.html

Of course, we also have some minor improvements :

• the new InOutContainer
• Wire mouse events
• some experimental layout options

You can view the complete changeset at:

http://javascript.neyric.com/wireit/VERSION.txt

And of course download the library from the home page: http://javascript.neyric.com/wireit/

Have fun !
Eric

The long awaited release of WireIt is finally here ! Awaited indeed, since a major bug was still present in version 0.3.0, preventing it to work on IE. It has been fixed for a long time on the repository but has discouraged many people using WireIt (hopefully not too much…).

What’s new in version 0.4.0 ?

• inputEx integration: inputEx is another of my YUI libraries, to create fields and forms. Through a new class called “FormContainer”, you can now build WireIt containers using the advanced forms options from inputEx. The value is saved at the layer level for you through Layer.getWiring

• WiringEditor: This is my favorite feature. While still largely incomplete, the Wiring Editor provide a full-blown interface to create visual languages. It takes care of most of the interface you might want to build a visual editor on top of WireIt. The language definitions are written in JSON, I hope this will greatly reduce the learning curve of WireIt.  The transparent ajax communication with the database is handled through a SMD webservice using YUI-RPC. WireIt provides a simple php backend that stores the JSON wiring states in MySql (used for the demos on wireit’s website).

• new options:
  • terminals position in containers: {top: , left: , right: , bottom: } instead of [x,y]
  • Terminal.alwaysSrc : force this terminal to be the source element in every connected wires
  • Container.preventSelfWiring : prevent connections between two terminals of this container

• Four Wire methods for handling mouse events. You can override them for a custom use :
  • onWireClick
  • onWireIn
  • onWireOut
  • onWireMove

• API Documentation with YUI Doc (much more precise on properties and events)

• Layer Map is now clickable and draggable. It will move the linked layer accordingly.

• IE Bug fix ! (finally)

• Moved to github for git repository and better wiki

It’s now pretty easy to build visual languages applications. (or at least, prototype them…).

You can follow some projects on http://github.com/neyric/wireit/network. I’m especially looking forward to LeifW XProc Visual Editor

Here is a little presentation that I published on WireIt’s homepage :

0.2.2 release

Although we didn’t add any new field in this release, inputEx 0.2.2 still brings a new set of options and bugfixes. (see the changeset for details)

The most significant part of this version, which I’m pretty excited about, resides in the following components :

• improved JSON-schema support (example one and two)
• yui-rpc: A Service Mapping Description (SMD) client for YUI, based on the SMD Proposal
• inputex-rpc: automatically builds a form from a SMD
• JsonTreeInspector widget: simple widget to display a JSON tree
• The SMD-tester utility

YUI-RPC or “What the heck is Service Mapping Description (SMD)” ?

Abstract from the proposal:

A Service Mapping Description (SMD) is a JSON representation describing web services. An SMD can defines the various aspects of a web service such that clients can coherently interact with the web services. An SMD can be used by generic tools to generate interfaces, human and programmatic, to access available web services. A wide array of web services can be described with SMD including REST services and JSON-RPC services. The SMD format is designed to be flexible, compact, simple, readable, and easily implemented.

SMD is basically a WSDL-like, but for JSON. It’s so much simpler to implement that I regret the time I spent on WSDL ! Instead of using XML, XML-RPC and SOAP, we are now able to play with technologies better suited for the browser environment: JSON, JSON-RPC, REST, JSONP etc…

Concerning the “programmatic interface”, the dojo framework already provides a remote procedure calls module. However, since I’m a YUI-user, I had to re-implement it on YUI, which gave birth to YUI-RPC.

How inputEx is playing with SMD

From the inputEx point of view, the best part of SMD is that the method parameters are defined using… JSON-Schema !

inputEx was already able to build a form from a JSON-Schema, so it was pretty trivial to implement an automated form generation to call webservices based on a SMD. Here is an example of such a form using a SMD for Yahoo! search.

I believe this will help a lot in decoupling interfaces and webservices. To prove my point, I quickly wrote some SMDs for popular webservices like Yahoo! search, pipes, Twitter, geonames, delicious, flickr, YQL and others, and built a tiny interface, the SMD-tester, to build the forms, call those webservices, and display the response.

Have fun !

We were so focused on developping very cool web applications with inputEx, that we delayed this released many times. Here it is !

Beyond the usual bug fixes, we received very cool contributions:

• MapField: Thanks to David Janes. It can handle MS Virtual Earth, Yahoo! Maps or Google Maps. It returns the latitude/longitude coordinates but also the zoom level.
• MenuField: Thanks to Maxime Réty. Useful solution using the YUI Menu widget instead of grouping dropdown menus.
• Italian localization by alexodus. If you speak Spanish or German, it will take only a few minutes to translate and post it on the forum.
• Json-schema referencing by Brad Clements. (see below)

Other things in this release :

• VectorField. I use this one for storing positions or any kind of vectorial information.
• FileField-beta. It only works if you’re using a POST method to submit the form, but might be useful for you.
• The inputEx Builder can now export the generated code to a page, and load a form from a inputEx Json definition or a json schema !
• We also moved the repository to GitHub, although the wiki has not moved yet.

Click here to view the complete release notes.

Have fun !

We received queries on the inputEx forum to switch our repository to Git.

The YUI library recently moved to GitHub, so I decided to move both of the YUI-based libraries, inputEx and WireIt, to the same plateform.

Here are the two new repositories: inputEx and Wireit.

The GitHub interface is friendly, it displays nice stats, has a cool API, and Git ! (But I’m missing a ticket system)

The good news is that it is really easy to import a SVN repository, and keeps the commit history.

See you on GitHub !

Ah… and Happy New Year !

inputEx 0.2.0 is finally available for download !

This version of the YUI-based json form framework brings a great set of fixes and enhancements:

• 11 new fields
• 3 new widgets
• 25 enhancements on previous fields
• cross-browser enhancements (IE 6 is on the way…)

I strongly suggest you take a look at the complete changeset, and our new list of examples.

Note: For those who were already using version 0.1.0, we wrote a description of the json API changes between 0.1.0 and 0.2.0.

Finally, I would like to thank:

• Maxime R. – for his bug fixing !
• Daniel Barreiro (alias Satyam) – for his help with YUI’s style
• mingfai – for his work on interactions and validations
• unomi – for his ideas, bug reports and forum help
• ClicRDV – for open-sourcing this library
• And many others – for bug reports

Have fun !

WireIt is back !

On June 19th, WireIt was removed from distribution for copyright infringements. After discussions with the Yahoo! User Interface and Yahoo! Pipes teams, a project started to provide the wiring widgets as a YUI module. (Note: I don’t work for Yahoo! so it is NOT an official annoucment…)

I’m very happy with this decision and I can’t wait having this integrated into YUI. However, the quality concerns of YUI requires very detailed attention to cross-browser support and YUI integration. It will take some time before it appears on the YUI roadmap.

Meanwhile, I’ve been contacted by many WireIt users, who where anxious about using code with copyright infringements, and about seing no more bugfixes/new features. Therefore, I decided to release this new version that does not contain any coprighted code/assets anymore !

If you use WireIt <= 0.2.1, you MUST upgrade to 0.3.0.

WireIt get sponsored !

Among WireIt users, Tarpipe decided to sponsor the WireIt library. I am grateful to them, for helping revive the project.

What’s new in version 0.3.0 ?

• Removed copyrighted images and code
• New drawing methods (straight lines and arrows, thanks to Marcin Iwa?ski)
• CanvasElement class (WireIt.Wire now inherits this class)
• Scissors class (for the scissors widget)
• LayerMap-beta (minimap to visualize/naviguate the layer)
• API Documentation much nicer and usable
• WireIt.Layer: “clear” and “setWiring” methods
• Bug fixes (including the much awaited positionning issue)

I also started an interactive presentation, to demonstrate WireIt features.

Have fun !