Comparison of using Ext AJAX calls and using Ext Direct for your data stores

So I had the time and figured I’d try out Ext.Direct and see what happened. A little background on what this is: ExtJS uses the traditional AJAX calls to communicate with the server, and that’s definitely the easiest way to execute calls. The issue comes up when you have a bunch of little calls that need to be executed on page load. For example, on the tool I'm developing I have 16 stores that each make their own call to ColdFusion to populate their data (little things like dropdown stores, and bigger things like stores that are used to populate the actual interface etc). After a certain number of calls that are executed concurrently, the browser itself will STOP making calls so as to not flood the server with hits (essentially stopping itself from being a mini Denial of Service attack). The problem with this is, depending on how many calls you make at once it can take a while to get everything needed back.

So Ext Direct is one method of getting around this. It sets up an API that allows for the database to accept a bundle of calls and then distribute those calls as necessary to the different functions on the server side. Then it bundles those results into one result back to the browser, getting around that browser restriction that they all have. There are other features but that’s the one I’m talking about for the timing breakdowns.

Anywho, I put together some metrics since I compiled everything from my master branch and then a separate compilation for my master-ExtDirect branch. I did a few runs each for refreshing application data (clearing the cache in ColdFusion), and then for just going to the page (no f5, just hitting enter. There is a difference ;)) Also Firebug uses 2 different times. One from total time executing  and another from page hit to ‘onload’ execution,.

Using ExtDirect: Total/Onload

• appRefresh Average: 1063.5ms/937.5ms
• Page Hit Average: 926ms/893.25ms

Regular AJAX calls: Total/Onload

• appRefresh Average: 3200ms/771.33ms
• Page Hit Average: 2770ms/883ms

So what this really says is that the onload times are pretty much the same for each, which makes sense since all you are doing is loading JavaScript files and I already have that compiled in one nice and pretty file (thanks to JSBuilder! But this is actually part of another blog post due to 'issues' around loading of packages and executing the addProvider function). The real benefits show up on the total time. Ext Direct is faster by almost 2 seconds in both scenarios. I know that doesn’t sound like much, but if it’s 2 seconds faster in our closed and ‘perfect scenario’ network, imagine how much faster it can be out there in internet land. Pretty slick stuff ;)

Odd behavior with ExtJS JSB file creation

I was compiling one of our projects to get it ready for an updated release and ran into an odd issue. Everything looked good on initial load but the page started throwing errors pretty much immediately. So I started digging around and saw that the JSB file that previously worked fine was only outputting 3 stores and their associated models. Whaa?

Turns out the developer had put a stores property with those 3 stores in the application definition function. It worked fine when loading everything 'manually' but when it came to the compile process it saw that stores property and apparently just STOPPED. Very weird.

So to fix it I just moved the stores definition to the applicable Controller and everything started working again. Very weird behavior but hey at least it was an easy fix (once I knew what the problem was of course!)

CFQueryReader updated, and missing DB column mappings!

So I use CFQueryReader from Steve Cutter and he has recently updated his reader to support the latest and greatest from ExtJS (4.1 to be specific) which is fantastic because we were hacking the crap out of his previous version to get it working how we needed it to be. Updates include tying directly into the model creation process, which means all you have to do is build the fields and you are set.

So I plopped it in there and it sorta worked. Sorta? Yea, some fields were showing up but others weren't at all. Very confusing. After much testing/debugging (this is the first time I dug through his updated reader so it took a little bit to figure out what was going on) it turns out that if you use the mapping feature to map database columns to your model columns, they get overwritten by the Array reader and turn into numbers vs your DB column names. Minor details right? Well not really. There are some plugins out there that depend on that being there. Bryntum's Scheduler does this (not their fault that the reader did something odd) but I found a workaround so it's not the end of the world. It's really only 3 lines of code that you have to add (but then you have to update your models to reflect the change. I'll get to this in a second.) 

//Around line 189 add this:
nameProperty: null,

//Around line 228
if(pos >= 0){
  fields[i].mapping = pos; //This was a long string but he already assigned it to pos so I'm using it here
} else if (this.nameProperty && fields[i][this.nameProperty]) {
  fields[i].mapping = Ext.Array.indexOf(obj.COLUMNS, fields[i][this.nameProperty].toUpperCase());
}

The nameProperty is right there with the ExtJS Writer object (default is 'mapping' which just so happens to be overwritten!). In the writer it is used to take custom mappings and tie them back so your database doesn't have to remember the frontend model names. So once you add that code to the appropriate places what do you have to do to your models? Well, get rid of the mapping property and instead use ANYTHING else. I use nMap myself and it works like a charm. But you have to add your new nameProperty value to your model and in the reader so your model should look something like this: 

Ext.define('Your.Model.Here', {
  extend: 'Ext.data.Model',
  requires: 'Ext.ux.CFQueryReader',
  proxy: {
    //...
    writer: new Ext.data.JsonWriter({ 
      //...
      nameProperty: 'nMap'
    }),
    reader: {
      type: 'cfquery',
      nameProperty: 'nMap'
    }
  }
});

Once you get to that point you should see your column data show up again and when you update your model data those same columns should be properly mapped to your database names! Amazing. I let 'Cutter' know about it but haven't heard back so we'll see if it's something I did wrong or if it's an actual bug.

ExtJS Socket.io/CF Gateway Code released!

So, very exciting news! I got released to put my code out there as an open source project. I was able to release just about everything I had put together on this. The Socket.io ExtJS 4 connector. The Socket.io ExtJS Grid plugin (to allow your changes to be pushed to all users!) and the ColdFusion Socket.io Gateway (including SSL support thankyouverymuch). Very exciting stuff. You can check them out at: https://github.com/softwarezman and let me know what you think! I'm open to any/all suggestions so please provide feedback!

Setting up SSL for netty-socketio / ColdFusion gateway

So I was disappointed to learn that my beloved netty-socketio package couldn't do SSL stuff out of the box. Very disappointed! My customer requires the use of SSL which means I can't release my fantastic capability without it. Except we must remember that I am not very good with the whole Java thing. I officially finished my Java classes 4 years ago now and have only used it one time since then (and that was a fantastic disaster!) So I was not looking forward to trying to piece together the code I'd need to make Netty/SSL work with netty-socketio. The stupid thing is I forgot exactly how Java uses the extends functionality. (Not inherits, even though I should use it... I am just trying to get stuff working at this point.)

I originally thought I had to build out this big structure overriding entire objects from netty-socketio just to get it maybe-kinda-sorta working. Turns out I didn't need to do any of that. I just had to extend the SocketIOPipelineFactory.java, and SocketIOServer.java files, in addition to adding the ServerSslContext.java file from netty's example pages. Turned out to be not that bad (so far). I really am just extending a few functions here and there to include the necessary links to the SSLContext object and HOPEFULLY it will work. It is at least failing where it's trying to pull the certs, which is where it should be failing. I'll have more tomorrow (once I figure out how to get Java's certs playing nice with everything else.) Looking good!

The pains of releasing code under Government development contracts

So I officially have something I could release for ColdFusion and Socket.io. Unfortunately I cannot (yet). Due to the nature of the contract I work under, I am bound by some fun rules that require not just my customers approval, but his bosses approval as well. And then after that I have to get approval from the contracting officer who is over our contract to get it release. I have to prove that it doesn't impact the government negatively to get their approval.

Should be easy right? Only 1000 lines or so of code and it supports messages, events, and JSON objects. Works pretty well with a few oddities tossed in there (of course). I'll try and release it ASAP but it's not going to be soon. I'm telling all those 3 people who happen to have come across my blog ;).

Functional ColdFusion Gateway for Socket.IO

So I got my ColdFusion Gateway working pretty nicely with Socket.IO. I have a sample running via Chrome, Firefox, and IE and it's going from the Socket.IO js package on the frontend to the netty-socketio package that I previously mentioned, to the CFC I built, and right back out. I tested it and I can send messages to specific users as well as broadcast to all users. I can send string messages as well as Json objects. Seems to do what I need it to do for now. I broke it out from the original cf-websocket-gateway package that I used as inspiration so that depending on the message type it will call a different function. I really only plan on using the Json message but figured I'd push it a little bit for those who might be interested in other functionality.
Speaking of which, I am trying to get my customer to allow me to release my package as an Open Source project to maintain. It's nothing fancy but I think it would help a number of ColdFusion users out there who want to try some new/cool stuff with WebSocket and Socket.IO. I know I am! My next task is to create a nice and modular ExtJS store plugin that can push store updates to all users who are connected. It would be a useful tool when having a large grid that needs to be refreshed from multiple users (who wants users to overwrite each others' data?) or even chart data like a stock market ticker (note to self, that'd be a perfect example to build!)
So that's my plan. I will try and release what I have to the outside world and I'll continue to develop some additional components to help interface with what I put together. Should be fun! Maybe one day I'll even get users looking at what I've done. Hah. We'll see.