Did you ever go to a concert? Well, then most possibly you've been a guest of Live Nation Clubs and Theaters: A subsidiary of the Fortune 500 entertainment company Live Nation, Live Nation Clubs and Theaters manages a wide array of venues within the United States. But this is not what they're famous for, they also use Orchard :). Specifically all of Live Nation Clubs and Theater venues and brands, including e.g. House of Blues, The Aztec Theatre, and the Coca Cola Roxy are part of a 50-site multi tenant Orchard application. And we helped to iron out a few issues. Running 50 sites in a multi tenant Orchard app is not an extreme number (we have 3600 sites on DotNest) but if you have such high-profile websites it's still a challenge, especially operations-wise, to keep it all working smoothly. Live Nation Club and Theaters's development team got in touch with us to help troubleshoot some evasive bugs that hogged the whole application randomly when modules were enabled or disabled. We drilled into the issue and came up with a solution for the application's hosting environment, so the whole platform is much more stable now. We also helped the Live Nation Clubs and Theaters team with a few other issues and questions, and still continue to work together so Orchard serves them well. This is what Bryan Pritchard, Sr. Director Digital Development & Content Production told about us working together: There was this pesky problem that surfaced randomly when we enabled a feature on one of our sites and caused recurring (though short) downtimes. Since there was enough work on our plate already we reached out to the Lombiq team to help us fix the issue, which they soon did! We'll continue to be in touch with Lombiq as our Orchard experts. Do you also have a big multi tenant Orchard app that needs some maintenance? You found just the right team, get in touch with us!

Recently the maintainers of the big Greek news site Parapolitika, the guys from the Greek subsidiary of Tatchit contacted us asking for our help: the site was going down routinely for some reason after going live (it was rewritten on Orchard from the legacy engine). The Orchard application was sometimes using up all of the server's CPU (despite it being a 24-core beast) and crashing the IIS worker process in the end. This needed some urgent fix because websites tend to be only worthy if they're alive... We immediately jumped into the task of getting the site stable! Neither the Orchard logs, neither the Windows Event Log revealed anything interesting. However soon we could experience the phenomena live: the worker process was eating up memory until at around 3,8GB while the CPU started spinning like mad and finally the process died. The Event Log told that ImageResizer.NET was running out of memory. Seriously? There are 32GBs of it, damn it! The culprit was the worker process running on 32b, thus not able to use the whole huge memory. While such big memory usage is not something Orchard does everyday (a vanilla Orchard instance in a 32b worker process uses about 80MB) this solved the immediate issue quickly. Together with some other tweaks to the server config the site was now running stable, quickly reaching new uptime records (although the previous uptimes weren't too hard to beat). In the newly gained peace we finally upgraded the site to Orchard 1.7.1 from 1.6 (the new version doesn't only give many features but also performs a way better). Meanwhile we also fixed an issue that could cause OutputCache to serve expired content. To quote Sotirios Roussos, CEO of urbanIT whom we worked with closely on this emergency: "After making some not demanding sites using Orchard, we decided to use it as CMS for creating the new parapolitika.gr, a really huge news site with more than 100.000 visitors daily and over 20 editors and a lot of content. It was a challenge for us and Orchard as well. Unfortunately, the first days were tough. Sudden breakdowns of site were appeared and the pressure was high. Orchard seemed to have limits, or maybe not? That's why we asked help for Lombiq, due to their experience into Orchard infrastructure. Fortunately, they did respond quick and spent hours and nights with us. Until we reach our goal. A stable and quick site. And, we did it. Thanx Lombiq! Keep up the good work!" It was a rush but we're really glad that we see a happy ending to this story!

On Friday we completed a short but important emergency assignment: getting the GI Joe Search & Rescue team's website back online. The website was moved to a new hosting company but wasn't feeling all right (the standard IIS 404 page greeted the visitors). This was especially problematic given that the team needed the website for their communication on the weekend. That's when we got contacted on behalf of the team and started investigating. After checking the logs and verifying the integrity of the deployment we recognized the source of the error: although the site's database was seemingly restored all the tables were empty! Running the restore again fixed the database and thus solved the issue altogether. Naturally we didn't charge the team for this emergency fix: they don't charge you when you need their help either!