Like module 16, I was invited into the Module 17 closed beta. This time it was organized a little differently to the previous closed beta and for those curious, this article aims to go over and highlight what changed and what, in my opinion, can be improved.
First off, instead of dealing directly with the systems designers, our interaction was with Lassor. Lassor came across as a lot more relaxed then the developers involved in the module 16 playtest, but that was likely just a result of it being a less hypercritical content release. Lassor has left a strong positive impression with me, it felt like he is a really approachable person, and very involved personally, for example, he offered to come in on the 4th of July to work in order to spectate any runs of the new trial that we did. This has continued past the end of the closed beta, with him often spectating trial runs on the public play test shard.
Now, a few points about the process:
1. This closed beta was better planned than the module 16 play test up front. During the module 16 play test it kind of felt like we were thrown in a sandpit and told to have fun, where this time going into this we had some clear idea of what they wanted tested (the new race, the new appearance system and most importantly the new trial) and more or less when changes were to occur.
2. Patches were a lot more frequent, happening every second or third day, so we got to experience changes much faster than in the module 16 closed beta. Allowing testing the changes and trying them instead of just waiting a week for the planned changes to be available for testing.
3. This time, most of the communication occurred via voice communication with ‘meetings set up at specific time slots. Whilst this is not necessarily a bad thing, to facilitate this the participants need to be available at the set times but many of those of us who were invited were from European time zones and most of the runs (and discussions) were organized at times that were simply not feasible. There were time slots set aside ahead of time where the developers would spectate on our runs and many of us were simply not able to make most of those times. What I would suggest is when scheduling time slots like this, find out where the people you are inviting are located, ahead of time, and if possible, try to accommodate for as many people as possible.
4. As a result of the voice communication, there was not much written communication and so for those of us who were unable to make most of the scheduled runs due to the time zone differences (for example, being at 3 am for us) it felt like we lost out of a lot of the communication process as a result. I think there should have been a bit more back and forth in written form, and not tipping it too far onto either side of the scale. A mix of both forms of communication, verbal and written can be beneficial. Some discussion of ideas, particularly in relation to trial ideas, what could and could not be done, and things that were willing and not willing to be added, would have been good. Perhaps using a discord text channels would have facilitated quicker communication than forums, and a middle point between both ends voice and forums.
5. In my opinion, the number of people that were invited to play test this module was too small, especially considering we were trying to test a 10 players raid. It made group organization difficult and one of the time slots had to be abandoned as a result. This was also an issue in M16, with the need for people who main specific classes, available at convenient to everyone times the pool of available people was just not enough, hindering group content testing.
6. Tested content and invited testers – It is my belief that more care should have been put into the testing group selection. In this case, the trial is aimed towards endgame players, it would have made sense to focus more on inviting players from that demographic then from players across the spectrum, as it would provide more focused feedback from the target audience it was aimed towards. The testing group should match the content that should be tested, preferably with testers that show interest, and can test multiple aspects of the new content. In this case, optimally, using end-game players who are interested in Fashion and played PvP.
7. Similarly with content creators, while I understand that many of the invited players were invited due to being content creators for the game (including myself), I feel that it is important to invite more specific content creators that are relevant to the tested content. If a content creator is unlikely to play the content when it reaches live, does it make sense to invite them?! Considering that they and their audience are not likely the type of players that are interested in that content to begin with. If a new module was entirely dedicated to the Foundry for example, I don’t think I should be invited, because that isn’t the type of content I create and I am not the best person to review it.
As far as the module itself goes, while it is small, I am also extremely positive about it. A lot of effort was put into making it and it shows. I would rather have small, well polished modules than big messes which needs to be cleaned up afterwards, and module 17 is exactly that, a small module where everything is very well done. The new raid is a shining example of what a raid should be and I hope we see more content like this in future. Whilst I do not engage much with fashion myself, the new system is a definite improvement. Over all, I consider module 17 to be the most well put together module this game has had to date in terms of polish and I think the team that made it should definitely give themselves a pat on the back.
For a module of this size, my opinion is that a closed beta was not as necessary, and it leads me to wonder why it was organized. Unlike module 16, I felt the closed beta for this module could have ended sooner. I felt it was in a good state before it reached preview. Perhaps it was done to experiment a bit with closed betas in general, to find a format that works better for them and improve upon issues that arose in M16. I do hope they find one that does work as Module 16 suffered due to issues with preparation and organization of the beta and if they intend to continue the closed beta program, it makes sense to find a format that works before they attempt another release on the scope of Undermountain or just improve upon the system to fit smaller mods. Either way, I am glad they did not abandon the project with the leaks that occurred in module 16, I definitely feel that the presence of the closed betas is a net positive for all.
That is it, I hope that this answers the questions of anyone who was interested in what took place in the closed beta, if not, feel free to ask in the comments.