_*Update*: I stand by most of the information in this article, but I have learned a couple of things in the last two month. I now recommend against pulling information directly out of $_POST or using form_builder. My Ahah Forms package can make a lot of this easier. Please checkout Secure Dynamic Forms and Subforms, Ahah_forms reloaded, and the examples in the Ahah Forms package._

Goal
To create a form widget that consists of an unlimited number of subwidgets, expands without needed a full page refresh, and can be easily added multiple times into any Drupal form.

Plan
Use hook_element to define the element, hook_nodeapi to handle the db interactions, and jQuery-based AHAH code to do the incremental page updates (AHAH – Asynchronous HTTP And HTML , not AJAX b/c there is not XML involved, and people seem to think AJAH isn’t sexy enough).

Approaches Rejected for attaching element to form

1. Using hook_form_alter and nodeapi, like upload or event – rejected b/c you can only add a single field to a form with this method.
2. CCK – currently there is no great way to deploy cck based solutions (import/export has been added, so this problem is probably going away).
3. Just += widget_create(), which works fine, but doesn’t seem to be The Drupal Way

Defining the dynamic Widget element
We start by defining out new widget element with hook_elements() (To follow along with the code, download the module as a zip).

I would love to define the element’s #process function here, but unfortunately we need the $node to do the expansion when we are doing an edit of an existing node with subwidgets attached. So we are forced to add our element into forms like this:

Sadly non-elegant. Hopeful there will the a more aesthetically appealing way to do this in the future.

Now, when the main form gets built, widget_expand will be called. The widget_expand function, is responsible for doing some bookkeeping: it attaches the JavaScript files; marks the node type as containing a widget; builds a tree in the form for the multiple widget names; and builds the wrapper for the dynamic bits. Then it calls _widget_build_widget.

This bit took me a while to work out, it creates a tree in the form that grows for each widget that gets added into the same form.

Similarly, this code builds the same array and passes it to the JavaScript (and handles getting the basePath to the js):

The _widget_build_widget function is where our dynamic widget actually gets built. It will also be called by the JavaScript, so it needs to be discreet from the bookkeeping that only gets called once per full page load. This function builds the widget element in four steps, each broken out into a helper function:

Often useful Form API Reference: (http://api.drupal.org/api/5/file/developer/topics/forms_api_reference.ht...)
Helpful handbook page: Adding a custom element type & expanding elements (http://drupal.org/node/37862)

Incremental Form Updates: AHAH!
Now it gets interesting. We made sure our JavaScript got added to the page in widget_expand. Let’s see what widget.js does.
Attaching the JavaScript events

Last things first. The last line of the js file is:

This gets run automatically as soon as the page is ready to go. In this context “attach” mean binding a jQuery listener to the widget UI elements that we want to respond to.

Widget.attach_all just iterates over the widget name array created back in widget_expand:

Widget.attach_widget is responsible for binding each widget’s form elements:

The hard work was done in the form generation, to make sure each form element had an id or class that allow us to attach to it. (One thing to watch out for with JQuery: $().click( fn ) registers a listener, but $().click( value ) actually triggers the event). I got bit by a bug trying to bind to the button’s id, but was able to use the class name. For each event type, we register the same listener. Everything the widget changes we are going to regenerate the entire widget and swap out the old view.

After all this setup, in Widget.update_widget we finally get to do our slick AHAH magic:

Here we gather up the parameters from the form (fromToArray), add in the name of the widget that has changed. Then we post back to Drupal, which returns the html for our updated widget in the data variable. Then jQuery .html method is used to replace the old widget with the updated html. highlightFade is a jQuery plugin to give the traditional AJAX “look at me, I have changed” fading yellow flash. Then it is essential that we reattach listeners to the new html.

Generating the Replacement Widget
We have covered generating the initial widget when the form page is originally created, and how the JavaScript swaps out the old widget with the updated one. But we still need to create the html for the updated widget, back in the widget.module.

First we need to declare a callback in widget_menu to from the path “widget/widget_update_js” to the function _widget_update_js(), which just pulls the needed info out of $_POST and hands it to _widget_build_widget_js:

This function creates a tree, but without the div.widget_wrapper that is created in widget_expand. The jQuery.html call replaces all the html inside the div, but leaves the div alone. The form_builder is called to do it’s behind the scenes magic, and drupal_render turns it into html, suitable for returning to the jQuery.post that requested it.

Connecting up the database

Widget_expand handles all of the form interactions, storing the subwidget data in the post between requests, but eventually that info needs to get stored in the database. This is where hook_nodeapi comes in. It’s a little clunky moving from the formAPI side of the world back to the node side, but we need a nid to store the info. I am not going to talk about this bit much, there are plenty of other nodeapi resources on the web. The only tricky bit here is dealing with the deep array. Insert flattens it, and load re-expands it. Also, I am being lazy on update and just doing a delete all old and reinsert all new.

Hook_nodeapi docs: (http://api.drupal.org/api/5/function/hook_nodeapi)

Drupal Bugs & Limitations

1. <?php
$form[‘foo] = array( ‘#type’ => ‘button’, ‘#attributes’ => array( ‘id’ => ‘my_button_id’ ) ); 
?>

   produces incorrect HTML. It kicks out an input with two id’s, the one I requested, and the default id. I need to see if there is an open ticket for this! Update: nadjo pointed me to the form #id property. Yay!

2. <?php
$form[‘foo’] = array( ‘#type’ => ‘item’, ‘#attributes’ => array( ‘id’ => ‘my_item_id’, ‘class’ => ‘my_item_class’ );
?>

   does nothing – probably just a documentation issue.

3. <?php
#process => array( ‘fn’ => array(‘arg1’) );
?>

   callsback to function fn( $element, $unknown, $arg1) – I haven’t been able to figure out what the $unknown is or why it is there.

4. It would be great if there was some flag to signal that your element wants the node in hook_element, so the #process is not needed by the including form.
5. I include the jQuery form plug in, just for the formToArray function. This makes posting back to Drupal much easier, and something like it would be a good candidate for inclusion in drupal.js

Final Thoughts

All the pieces are in place for creating JavaScript-enabled dynamic forms in Drupal, and it isn’t rocket science, but it can be very fussy work. You need to hold multiple models of the same data in your head at the same time – the formAPI array, the node object, the generated HTML and the JavaScript. It is very easy to get the nesting wrong or attach the JavaScript to an incorrect key. Being able to debug both the php and JavaScript while the page executes is key. I used Eclipse PDT with XDebug for the PHP, and the Firebug extension for Firefox to debug the JavaScript, and to examine the HTML in real-time as it changed.

It was roughly 10 times harder to get this working with multiple widgets on a page, compared to a single one. Keeping track of which widget was making a request, and getting that info everywhere it needed to be was the biggest challenge. Also, I probably shouldn’t have called the example widget, since that is the same term CCK uses, but such is life.