With the recent release of another IDE I am using, the Netbeans Java IDE (an Open Source project sponsored by Orcale, http://netbeans.org/), I started to compare the various beta-programs I am involved and essentially what works and what doesn't work. I'd like to share these thoughts and bring them up for discussion with my fellow REAL Studio users.

First off, I'd like to point out that I have not been involved in any REAL Studio betas since the release of 2009r5 which I considered such a failure so I didn't renew my license and thus have no access to RS betas anymore. Also the fallout of the 2009r5 release that required the 2009r5.1 release to fix bugs to me made absolutely clear that the beta-process is not working as expected.

As a programmer and extensive user of some of the tools, I feel the need to help the developers of the tools to provide me with the best experiences using their products. Beside developing in REALbasic, I create software in PHP, JavaScript and Adobe Flex so I have three other IDE:s installed on my computer; NuSphere PhpED, Netbeans IDE and Adobe Flex Builder. Other tools I use and actively provide feedback to the developers and the community are Sencha (formerly Ext JS), Apple iOS, EditPad PRO, QNAP, Mozilla Firefox and Sonos. Most of these vendors are open with the development process and provides public or invitation-only betas. All of the projects has a web-based forum where discussion of the betas are handled and where the developers (or committers of Open Source projects) are extremely active. The forum way of communicating makes it easy to search and see if the problem is new or potentially already solved and contains the full history of the beta and the forums of have a complement Wiki where mission statements and roadmaps are posted. De-facto standard bug-tracking tools (for example Mantis, Bugzilla or JIRA) are used to track issues and feedback.

The recent release of the Netbeans IDE (version 6.9) featured a very nice addition that I have never come across before in such a structured way that I thought it was worth discussing. During the process of creating the Release Candidate (RC), a Community Acceptance Survey was made so the developers could get a feeling of the quality of the release. Seven simple questions were asked and the questions relevant to this discussion were:
1.    Can you compare NetBeans IDE 6.9 and 6.8 in terms of Overall Quality?
2.    Can you compare NetBeans IDE 6.9 and 6.8 in terms of Performance?
3.    What do you think about the Editor improvements?
4.    What are the 3 top features or improvements that you would like to see implemented in the next release?
5.    Do you have any other comments on the IDE?

The first question had the option to list your own show stoppers that were then analyzed by the developers and evaluated with the result provided in the form of a Wiki-page. All comments to the questions are also listed. The survey is available here: http://qa.netbeans.org/processes/cat/69/ca_results.html

Netbeans 6.9 was released on schedule in mid June and contained major improvements in speed and compatibility. Netbeans' release schedule allows for a quick maintenance release (6.9.1) containing fixes for a couple of the show stoppers reported and a schedule for other show stoppers within 5-6 weeks of the 6.9 release.

So, after this rather length introduction I'd like to discuss the beta process of REAL Studio and some improvements that I would like to see that would attract me to resume donating some of my precious time to the development of REAL Studio and also to reactivate my subscription for REAL Studio.

RS Roadmaps - where art thou?
With each new release of REAL Studio we get a trickle of information during the late alpha process that is later turned into Niagara Falls of new features, fixes for old issues, fixes for new issues and other issues that has crept in during later betas. By that time in the beta process, the focus turns to into patching up new stuff as well as trying to avoid breaking existing functionality and then getting to the finishing line in time for the 90 day release schedule.

Virtually every other product I am using have a roadmap making it possible for both users and developers to see what will come in the new version. Some excellent examples of roadmaps are for Sencha and Netbeans, see http://www.sencha.com/products/js/roadmap.php and http://netbeans.org/community/releases/roadmap.html

Each new release of REAL Studio brings new surprises and increased delays of Cocoa (of which I personally couldn't care less of as I don't develop for the Mac OS). Having a roadmap would make it possible to see what is up ahead and also for RS to focus on the development of the product.

Stop using the RB Beta Mailing-list!
The only purpose of mailing-lists seems to be to fill up your mail-box and provide for unstructured discussions. Ideal would be a locked part of the RS Forums for participants of the beta program (like the Apple Developer Forums for example). This forum would be easier to monitor and also to find old information. QNAP (makers of SOHO NAS-solutions) have parts of their forums devoted to beta-testers and they are actively moderated by QNAP developers and QA people.

Structure the feedback process!
Feedback is recorded to requests that are marked as fixed but several users report them being not fixed and a new request is opened but not merged with the old one.  Requests 9035 and 12192 is nice combo of issues that was reportedly fixed in 2010r2 but wasn't and then a new request 12192 was created, disregarding the information from 9035 that was marked as solved!

Introducing an Acceptance Survey among the beta-testers would dot the i's and cross the t's of the release process and would make a very nice addition to get a feeling of the overall quality of the release.

To summarize, three items I'd like to see from RS that would make me consider joining the beta-program again:
• Roadmaps of features
• Using forums for beta discussions
• Structured feedback process (and possibly an Acceptance Survey)