Getting Started with the Benchmarking Framework

[nickburch]

My Alfresco setup seems to be having a few performance issues, so before I dive in and make changes, I want to take some benchmarks so I can ensure I make things better! Having seen the <a href="https://devcon.alfresco.com/berlin/sessions/alfresco-benchmarking">DevCon Alfresco Benchmarking talk</a>, and reviewed the <a href="http://www.youtube.com/watch?v=ahc6v6zqhwQ">video recording of it</a>, I felt that the new (but not yet finished/supported) Benchmark framework would be the one to go for.

I tried asking support nicely, but they said it's not supported and I'd be on my own. So, I checked the relevent bits out of <a href="https://svn.alfresco.com/repos/alfresco-enterprise-mirror/benchmark/">SVN</a> (enterprise svn login needed), and did a build of the various parts. This sucked down lots of pre-built bits of the benchmark framework from <a href="https://maven.alfresco.com/nexus/content/repositories/snapshots/org/alfresco/alfresco-benchmark-clie...">the Alfresco Nexus server</a>, so non-enterprise customers might be able to grab it from there, but I'm not sure…

Next up, as explained in the <a href="https://devcon.alfresco.com/berlin/sessions/alfresco-benchmarking">DevCon talk</a>, the framework needs Mongo and ZooKeeper. I installed those, got them working and running (plenty of tutorial on that elsewhere!). Then, I tried following the readme.txt in the client project, and built + ran the war via jetty. When that started, I went to <a href="http://localhost:8080/alfresco-benchmark-client/">http://localhost:8080/alfresco-benchmark-client/</a> in my browser, and got an interesting looking web interface, with lots of empty dropdowns. Don't be fooled by this web interface and the readme, you currently want the command line client! (I guess the web interface is still WIP). Instead, in your client/target/ directory, you'll spot a onejar version.

java -jar alfresco-benchmark-client-1.3-SNAPSHOT.one-jar.jar‍

will bring it up. When running, it's fairly clear what to do, and there's some help, but be aware that it isn't all that forgiving of extra spaces, doesn't have up arrow etc. An initial "ls" will give an empty set of clusters, but don't worry.

With a client going, next you need a server. Again you can build a war, but that doesn't seem to work. If you've checked out from svn, you'll need to edit a properties file to tell it the version (eg edit default.properties and set project.version to 1.4.0). If you don't, you'll get a cluster called "${project.version}" and lots of things will get sulky about that! You can either start the server from the one-jar, or do "mvn eclipse:eclipse" to build and eclipse project, import that, and run BMServer. I'd suggest the latter, so you can debug it, as the server doesn't seem to give a lot of feedback on why things aren't really happening.

When the server is running, you should be able to do a "ls" in the client, and see a cluster with one server in it appear. "use 1.4.0" (assuming that's what you set the version to) will get you into an interface where you can do things. Pick a project from the tests subdirectory, build it with maven, then do "upload jar" to upload the jar (and again for any dependencies). You'll also need to do "upload configuration" to send up the default.properties and context file - see the end of the DevCon talk video. Finally, you can create a test run, load, and start

Unfortunately for me, at this point the server jumps into life, and then short reports the event processing stopped, without any errors. By running the server in Eclipse, and setting suitable breakpoints, I've been able to work out why. In the PrepareSites processor, it fetches from a mongodb field "mirrors.<servername>.users" (in my case mirrors.localhost.users) to work out what to create. That's blank, so no sites get created, so the tests have nothing to do.

So, I have two questions:
* How do I go about setting these mirrors.??? values? Via the client? Via something else? (I can't see anything on it in any of the readmes)
* Could someone on the benchmarking team tweak it so that zero users in PrepareSites gives some sort of warning, to help people in my situation realise what they've done wrong?

[dbiggins]

I believe the values are being set in mongo by the scripts themselves.  For instance, I think the user creation test creates the users in the alfresco repository AND the equivalent lookup records in mongo.  I used robomongo after I had run several tests, and there are collections for 'mirrors.testfiles' and 'mirrors.myserver.users'.

As a side note, I had no problems running the enterprise user creation test (alfresco-benchmark-bm-ent-signup-1.0.0.jar), but the share and webdav tests kept failing.  I couldn't see any errors on the client or server, but when I looked at the results collections in the mongodb (using robomongo), I was able to see the specific error.  After I cleared up the errors (the tests were calling methods whose names had been changed, for instance) things went smoothly.

Good Luck!