This is an article written for the Cincom news letter about the use of Smalltalk in the OpenSkills SkillsBase system.
The OpenSkills SkillsBase
OpenSkills is a global non-profit association. We exist to promote the skills of our members, and support them in their professional endevours. The association is entirely member funded, so OpenSkills can focus completely on the best interests of members.
The SkillsBase system is the primary vehicle for promoting the skills of OpenSkills members. It is essentially a resume database into which any member can enter their details. The data model of the SkillsBase tracks that of the HR-XML schema, and indeed members can export their resume to an HR-XML file at any time. In order to make members skills as easily to find as possible, the SkillsBase may be searched by anyone at no cost.
As of 2004, OpenSkills has membership measured in the hundereds and spans 8 countries. The association is targeting a membership in the thousands over the next couple of years, and a membership in the millions in the longer term.
The SkillsBase, therefore, must be scalable and accessible. The system must also be able to evolve smoothly over time as the functional demands upon the system evolve.
Accesiblility is achieved using valid XHTML and CSS. The SkillsBase is a web application, and you can see it in action at skillsbase.openskills.net.
The runtime SkillsBase system is made up of the following components:
Using this architecture we have minimized the amount of work we have to do. We had to write the business logic (create the model classes), and we had to write the code that responded to web requests.
- GemStone (www.gemstone.com) is the operational repository and application server environment. Within GemStone, we run:
- Our application logic (model and XHTML rendering)
- Hyper (HTTP Server)
- Squid (squid-cache.org) used as a reverse proxy. Squid takes all the hits from Internet and:
- Acts as a cache. The Smalltalk application adds headers to every response indicating how long a page is valid. This is a big win for a public facing web service. For example, some pages have a cache life of just a few seconds, but if we suddenly get thousands of people try to hit one of those pages we know that the application server will only be hit once every few seconds, rather than thousands of times a minute.
- Load balancing. Squid assigns each request (which could not be satisfied from the cache) to a specific http server in a simple round-robin fashion. The SkillsBase can (currently) have up to 2000 server instances, the limit being the number of gems GemStone supports.
- HTTPS unbundling. We can have connections over the internet be secure using HTTPS, but our application server does not have to handle the HTTPS protocol because Squid "unbundles" the simple HTTP request before passing it on.
... and we don't have to build our own solutions for:
- object -> relational mapping issues - GemStone does that.
- making the application thread safe - Each gem handles one request at a time. If we need more capacity, we run more gems.
- security over the wire - Squid implements authentication and https unbundling
- load management - Squid does double duty here. As a cache, Squid limits the number of requests hitting the application server. As a load balancer, Squid spreads the load among the gems that comprise the production system.
- scalability - The combination of GemStone and Squid means that the system can grow from having one or two http server gems on a very small server, up to thousands of gems on a multi-way system.
The Development process
All development work on the SlillsBase takes place in a Cincom VisualWorks Smalltalk environment.
The objects that make up the system run in both VisualWorks and GemStone in the same way, thanks to a portability layer. The cost of creating the portability layer has been repaid many times over through our ability to develop and test the system using the superb VisualWorks environment.
The development lifecycle goes something like this:
- Determine the features to be added (& bugs to be fixed) in the next version of the sysytem.
- Load the live version of the SkillsBase system (including all dependencies) in a single load action from our StORE repository.
- As far as possible, build SUnit tests and modify existing tests to define how the system is to behave in the new version.
- Start the development cycle of running tests, adding/modifying code and publishing progress to the StORE repository.
- Once the system is performing per the requirements for the new version, create a completely clean GemStone database and move the application over - this is achieved with a single do-it. Test the system and fix the inevitable one or two porting issues, publishing the changes to StORE. Repeat this step until the system runs identically in GemStone and VisualWorks.
- Restore a production GemStone database dump to a clean database in the development environment and write any necessary scripts to migrate existing persistent objects to the new versions. Test the upgrade script(s). Repeat this step until everything works smoothly.
- Backup the live GemStone repository, move the new code to the production datbase, run the upgrade scripts & commit. (Note that the production system was never unavailable!)
- Go back to step 1
We really have stood on the shoulders of giants to create the SkillsBase system. It's been a kind of aggressive laziness. By using a language as flexible as Smalltalk, a development environment as powerful as VisualWorks and deployment tools as capable as GemStone and Squid, we have been able to build a highly capable, scalable and robust system for the minimum of expended developer effort.