• Webform
• Product Browser
• Reciepe Collection
• Site Search
• Twitter Integration

For over 25 years, Stuwalk Energy has supplied and operated a vast network of public and private cardlocks in remote areas of Northern BC. They offer both traditional and unique distribution methods—fleet trucks, rail delivery and off-road bush trucks—that get fuel where it needs to go. Stuwalk focuses primarily on the natural gas extraction industry, but also serves the fuel needs of local communities.

• Responsive design
• Site rebuild
• Product catalogue
• Map view for all locations

After having some difficulty migrating to Drupal, American Institute for Economic Research (AIER) enlisted Drupal Connect's help in redeploying their website. AIER used Joomla! for their previous website, whose content needs to move to their new platform. AIER also sells print books and accepts donations. Some donations grant access to certain AIER membership levels, which have various perks. Constant Contact is used for email subscription management and marketing.

Solutions

To make things quick and easy, Drupal Connect migrated all of their content over with the same categories and users. The "Archive" items are a collection of PDF files (many scanned and OCRed) with content dating back to 1937! This content was imported separately to keep things clean and intact. Now that everything has moved over, AIER intends to further refine their content to better the user's overall experience and searching abilities. Ubercart was used to allow for easy scalability and automation of certain purchases. AIER now has the ability to configure donation membership tiers with automatic Drupal Role assignments. Drupal's Constant Contact module was quick to install and configure. Several Views were created to make content displays and reports easier to manage.

In the early fall of 2010, DrupalConnect began working on Waste Management Inc.'s high-profile, high-traffic collection of recycling and green-living sites, collectively under greenopolis.com, as part of a long-term reorganization and revitalization effort.

ThinkGreenRewards.com

Drupal Connect recently redesigned Waste Management's new, flagship community recycling program, Think Green Rewards. ThinkGreenRewards.wm.com has a beautiful, modern design with an intuitive UI that makes for a pleasant UX across the site's broad demographics. DrupalConnect employed all of the latest theming techniques with jQuery, image sprites, and css3 on top of a 960 grid with a stylish font via cufon for an elegant and bold Web presence.

Theming Technologies

• Custom theme
• 960 grid system
• jQuery
• CSS3
• Image Sprites
• Proprietary font usage via cufon

Greenopolis.com

We hit the ground running, committing code early on, and put out our first release within two weeks. We brought in additional resources to facilitate top-notch and on-time deployments, focusing on developing new code and fixing and enhancing old code. Two main areas stood out as great concern for our client:

Development Milestones

• Major refactoring to the backbone of the sites—the consumer recycling transaction processing
• Automating the integration of recycling data from Waste Management facilities
• Developing thousands of lines of new code
• Maintaining thousands of lines of legacy code
• Extending and customizing of Ubercart and views

High Traffic

• Pressflow 6.19
• Load-balanced web servers
• MySQL Database Cluster
• Memcached
• Gluster FS
• APC
• Varnish
• 20k visits per day

Drupal Connect was hired in December 2010 to assist with a major upgrade from Drupal 5 to 6 of a respected online technology industry magazine. The specific areas our team was tasked to address were performance and scalability issues, which required expert-level Drupal experience and were critical to the overall success of the project. This case study describes only some of Drupal Connect's work on the project, specifically that led by Lead Developer Ryan Palmer to address the performance and scalability issues.

Environment

• Apache 2.x, PHP 5.2.x, MySQL 5.x
• Memcached for Drupal cache and sessions
• Barracuda load balancer distributing traffic across two Web nodes
• Web nodes running NFS for sites/default/files. 16Gb RAM each.
• Dedicated MySQL server

Application

• Drupal 6.x
• 190+ modules in use (180 at launch)
• ~33 custom modules (12 features), ~55,000 lines of custom code, excluding features
• 1 custom theme: 14,000+ lines of CSS, 1,300 lines JS, 142 templates

Content

• ~150,000 nodes
• ~18,000 taxonomy terms
• ~10,000 users
• 25 content types

Throughput and performance design requirements

• 40,000 page requests per day
• Page generation under 1s per page

Analysis Tools

• XHProf - PHP profiling tool developed at Facebook
• jMeter -- load and performance testing. test profile built to measure performance and throughput based on 9/3 anon/auth mix for various parts of the site (article pages, answer pages, various landing pages). Uses a real session cookie to simulate being authenticated
• apache bench -- good for specific urls, measuring throughput
• Firebug/Google Chrome network analysis

Concerns

• Application performs poorly, inconsistently
• Anonymous page load time not acceptable

Application baseline

XHProf was used to improve baseline Drupal application performance by analyzing callgraphs and reports from various high-level administration pages. This takes the custom theme, primary menu, site-wide blocks, most third-party services and all content out of the equation and in theory should represent general application overhead. This was especially important considering the high number of modules used on this project, and the unpredictability of contributed modules and pre-existing custom code. Once slow or otherwise problematic PHP functions and processes were identified, steps were taken to optimize MySQL queries, cache third-party data services, cache views, cache objects (users, nodes, terms, etc.), or otherwise optimize them.
Per-path performance

Based on our jMeter testing suite, client feedback and first-hand experience, certain site sections were identified as being slow. Whether issues were persistent or intermittent, XHProf reports were captured and performance issues were addressed using the same methods described above.

Specific caching strategies:

Blocks: blockcache module was used to specifically set block cache settings, and block cache expiry times. This did require a patch to Drupal core. Example code for setting block cache expiry @ 5 mins in an update hook: /**
* Implements hook_update_N().
*/
function example_blocks_update_6000() {
// Force these blocks to expire after 15 minutes.
foreach (array('example_most_articles', 'example_most_articles_side', 'example_most_blogs', 'example_most_blogs_side') AS $v) {
variable_set('bc_life_example_blocks_'. $v, (15 * 60));
}
}
?>

Object caching: Memcached was leveraged for fast object storing, with smart expiries: /**
* Load the user object from cached values
* @param $uid
* The profile owner's user id or an array of values (name, mail, uid).
* @return $account user object
* Cache results for 60 minutes
*/
function example_user_load($uid) {
static $example_users;

$hash = md5(serialize($uid));
$cid = 'example_user:'. $hash;

if ($example_users[$hash]) {
return $example_users[$hash];
}
elseif ($cache = cache_get($cid)) {
$example_users[$hash] = $cache->data;
return $cache->data;
}
else {
$account = user_load($uid);
cache_set($cid, $account, 'cache', time() + (60 * 60));
return $account;
}
}
?>

Views caching: for certain problematic views, it became preferable to clear caches based on node authorship and administrative actions rather than time-based expiry. Query and rendering cache times were set far higher than usual (1hr or more) and hook_nodeapi, hook_form_alter and submission callbacks on administration interfaces were used to clear views caches: /**
* Implementation of hook_form_alter();
*/
function example_articles_form_alter(&$form, &$form_state, $form_id) {
switch($form_id) {
case 'nodequeue_arrange_subqueue_form':
if ($form['#subqueue']['qid'] == 3) {
$form['#submit'][] = '_example_nodequeue_arrange_subqueue_form_3_submit';
}
break;
}
}

/**
* Submission callback to clear a cache upon nodequeue 3 form submission
*/
function _example_nodequeue_arrange_subqueue_form_3_submit(&$form, &$form_state) {
cache_clear_all('views_default:editor_picks:en');
}
?>

Content delivery optimization

Where possible, all standard Drupal measures were taken to optimize front-end loading performance. Pre-existing circumstances meant that JavaScript aggregation was not possible, so steps were taken to amalgamate files manually. Anonymous page loading performance was determined to be unacceptable, largely due to the module overhead of the application. In fact, a near linear relationship was found between the number of modules enabled and the baseline performance of the application. The suggestion was made by Drupal Connect at the 11th hour to implement a reverse proxy cache (Varnish) to cache anonymous pages plus static assets to both dramatically improve anonymous page loading times and take the load off the very heavy apache processes for the serving all static assets. Varnish was installed by the client and configured by Drupal Connect, to operate per front-end server and between the load balancer and apache. Varnish was configured to cache aggressively and effectively, and Drupal was optimized to send proper headers, avoid sessions for anonymous users, etc. The end result was an average 95% "cache hit" rate and an anonymous page load time of 30ms (front page tested with ab, 95th percentile, n1000, c5), considered by all involved to be an exceptional outcome. The memory footprint of Varnish was large (2Gb+ per server), but CPU usage was less than 1% on each machine even while serving 95% of all traffic.

Lessons learned

• More modules = slower performance. The relationship was found to be nearly linear for baseline performance.
• Caching strategies should not be an afterthought. Views & block caching should be enabled from day one.
• Sometimes Drupal core needs to be hacked to achieve aggressive caching objectives. Drupal 7 should be more robust in this regard.
• Where practical (memory does cost money), Varnish should be used to serve all anonymous traffic.
• BUT, Drupal must be built with the assumption that anonymous users must not have SESS cookies.