| abstract
| - Believe it or not, it has been over a year since NXT was released into the world! In the beginning, 50% of you started playing immediately, and since then we’ve observed a steady incline of conversion from the old Java client to the new NXT version. As possibly the largest project in the history of RuneScape, we had a responsibility to you – our community - to ensure the highest possible quality in the required time frame. We hosted two closed beta tests to gauge player reactions to the new client and we were blown away by your reactions, feedback and bugs. Over 30,000 bugs were reported during both weekends, that’s 60x the normal amount of reports we take over a normal weekend! To our surprise and delight, quite a few of our beta testers launched petitions on the RSOF and Reddit to keep NXT open to the public. This feedback helped us develop a timeline towards a release date. After a few meetings, a bugfix checklist and a month of running around the office like a Daemonhiem ferret, we released NXT. As bold and stupid it may seem, releasing NXT before “ready” was totally necessary. File:NXT - 1 Year on 1.png RuneScape is a gigantic game, it’s absolutely massive! On average it takes around 400 days to “complete”, 9,600 hours, and with an ever growing library of content it becomes unwieldy very quickly. Even with our entire QA team, it would take 2 solid weeks each to experience the entire game once between all of us. Alas, there’s more! We support Windows, Mac and Linux (to a certain extent). That would mean we need to run through the game two more times. If you include the epic range of vastly different GPUs, CPUs, HDDs, SSDs, Drivers and Operating systems types (XP, Vista, Mountain Lion, 32bit etc…) rotating all their possible combinations, this goes from 9,600 hours of testing to around 96,000,000 hours of testing. Unfortunately, I would find it extremely difficult to even live this long (10,959 years)! Despite the challenge we managed to get NXT to a point where around 90% of people could play without running into any crashes, which was considered “acceptable” considering we had the Java client as a backup. File:NXT - 1 Year on 2.png When people talk about automated testing, the first thing that comes to mind are Unit Tests. They're really great and reliable for ensuring that a certain portion of your code actually does what it’s intended to do. But we knew we needed a more practical approach for testing graphics-related features, and the wide array of graphics cards and operating systems. Our CompatLab (compatibility laboratory) is a continuous project that was started quite a few years ago, and due to new hardware coming out every year (and the fact that we aim not to deprecate older machines unless we absolutely have to), it has never stopped growing! We’re now at about 40 machines and for each NXT release we’d run tests on each of those machines at least once. We also run a thorough compatibility sweep for each graphics-heavy branch that goes in for testing. Done manually, this would be an exhausting task (more than a full day’s work for 1 person), therefore we automated the process by selecting a series of locations and actions that will thoroughly test a set of graphics features for NXT. Then, using an in-house system, we got each of the compatibility machines to communicate with a web client, which would tell these machines to run certain tests and come back with performance data and screenshots. All a tester would need to do then is browse through the results and see if any of the results could pin-point a potential problem. This freed us from doing routine-checks and gave us, in return, more time for exploratory and destructive testing. File:NXT - 1 Year on 3.png Unlike conventional QA, Tech QA works systematically different. Our bug priority does not follow the conventional Trivial – Blocker scale. NXT Bugs are classified by impact, visibility and request. An example of this would be:
* Crash when depositing in Trouble brewing on an Intel 520 card
* Dyed items aren’t as bright as Java
* Player disappears briefly after eating or drinking If you thought the crash would be fixed first, you’d be wrong. Up to 0.2% of our players suffering a potential crash from content hardly used is not a critical priority. If you thought the dyed items would be fixed first, you’d also be wrong. Your e-date outfit is not that important to the global population. ;) If you thought players disappearing briefly after eating would be prioritised, you’d be correct. This affects everyone and is very annoying and has probably been reported 200 times in the past hour. Issues that affect everyone, are extremely annoying and are reported by everyone and their pack yaks are normally the first to get fixed. This doesn’t mean we don’t care about the uncommon issues, actually, quite the opposite. Unusually obscure bugs tend to unravel core flaws in our systems allowing us make more impactful fixes and optimisations. So yeah, thank you for having obscure PC setups. File:NXT - 1 Year on 4.png We have plenty of tools at our disposal in NXT - we're quite lucky as Java didn’t have such an extensive library of graphical debugs, and these are used to help us identify bugs in the 3D game world more easily. We also have extended telemetry information in our “displayfps” command. Here is an example of some of our visually interesting debugs we use. Normals - Used to the direction that surfaces face, represented by colour. File:NXT - 1 Year on 5.png Emissive – Used to show which models are emitting light. File:NXT - 1 Year on 6.png Shadow capsules – Used to show the directions of shadow projections. File:NXT - 1 Year on 7.png Picking – Used to show the click boxes on models File:NXT - 1 Year on 8.png LOD Viewer – Used to display how models are affected by the level of detail system. File:NXT - 1 Year on 9.png To a person who does not know what they are looking at, this might look like a RuneScape modern art installation and I would agree, they look broken and noisy. I know what you’re thinking, “whomever can acquire information from this must be either a wizard or some sort of time lord”, and you would only be half right, I do have 99 magic (though I am still trying to work out this whole time travel thing). A while ago I was testing soft particles, which proved much harder than looking at various particles in the game. I found that it was almost impossible to see any difference, similar to comparing deep pink to magenta. Luckily a debug was added with the ability to change the severity of particle softness, making it so I could physically see the difference, and making testing so much easier! File:NXT - 1 Year on 10.png Recently, we’ve switched from a two-week release schedule to a monthly schedule. We saw fewer critical bugs cropping up and the rate of bugs appearing in Jira – our development tracking tool - was gradually slowing. Although the volume of new bugs has decreased we are still actively seeking bugs from your reports, Twitter, Reddit and the RSOF. There are still lots to do in preparation for the big Java switch off, but first, we plan to have most, if not all Java functionality in NXT before we even think of pulling the plug. I hope for a day to dawn in the no-so-distant future where the only excuse for not playing in NXT is because of Old School. Keeping on top of two codebases is extremely restricting! In the past two months, we have ninja’d features into the game without anyone but Sudo Bash knowing (*shakes fist violently*). Volumetric lighting and normal maps been fully integrated into NXT. I’m super excited for you to see them in all their glory come next week. We’re also working on some super top secret stuff which I dare not divulge, you will just have to wait to find out! It may be worth noting that our developers are not full time bug fixers, while we are aware there are still a fair amount of bugs in the game, features, optimisations, refactors and content requirements are still on-going and sometimes come before minor fixes. File:NXT - 1 Year on 11.png The RuneScape Technical QA team thanks you for your continued support, bug reports and suggestions. NXT has developed so much over the past 14 months and it certainly couldn’t have been done without you! You guys are awesome! Cheers, Mod Dolan & Mod Noldor Technical QA Engineers
|
![[RDF Data]](/fct/images/sw-rdf-blue.png)