Dear PDXs QA guys and girls, how do you work? (no rant, just curious)

[Bugnr01]

First of all i want to thank you for your work! I am sure you prevent a lots of bugs from geting released and its a ungratefull task anyways. If you make a good job no one will notice you but if you miss some bugs you will get blamed for it (at least from the custom side).

I personaly have never done QA or proper beta testing so i was curious how is the QA of PDX working.
Maybe i will understand better why you catch some of the bugs and why others slip through your net.

Because is hard to understand why bugs like a non working ethic drift in stellaris can pass unnoticed or new features dont work proberly like tributary break away two years after the peace deal in EU4. As a customer i allways think someone must have noticed this, or have they never tried to do that?
Edit: Or was it just labled as not show stopping and if how are you deciding what is and what not.

I asume you are not just playing the game and instead are geting checklists with "do this and check that". So if something slips through its more because it was not on a list rather than an oversight from a tester? And i would even dont blaime the list guys / girls for that (you cant allways imagine what players are going to do or what change can influence an other unintetioned part of the game)
I also asume its due to the hot code, something you checked off today as working can be broken tomorrow because of code changes?

So how does PDXs QA work?

 

[Vetgirig]

They actually stream their QA session live - and even have commentary:

Well thats only one part of their QA - but its one way to really stress test the game - by playing it. They got one rule about bugs. If you find a bug you may abuse it - once.

 

[Bugnr01]

I am well aware of the Dev mp clashs but they seem to be to me:

• at first to promote new features
• secound a small check of the balance of some features (as far as the mp setup allows it). You can not check the Daimyo Shogun balance if three players are just taking Japan apart.
  
• and just third to chatch some bugs if you find them, but you are not going to search for them in a compedetive mp (like switching religion as Japan just for the purpose to look how the isolationims mechanc is responding to it)

There have to be (and im pretty sure there is) much more time and manpower thrown at bug finding/fixing and balancing than this one multiplayer game!
And i want to know how they work. If it is possible?

 

[aruon]

prior to livestreaming dev MPs, they were always done without cameras and were more about debugging/testing than bragging rights. it's why so many people play them, so that someone will eventually spot a bug, or they'll realize a feature doesn't play well or stuff like that. (or the chat will break and whatever hilarious antics groogy is up to will fall on deaf ears)

and yes, there's much more time spent particularly by QA to debug outside of a gaming environment.

 

[DDRJake]

Perception of what QA is like from anyone not involved in the profession or industry is almost universally off the mark. Here's a fairly good thread written by HoI4 QA on the matter:

https://forum.paradoxplaza.com/foru...-iron-iv-development-diary-55-hoi4-qa.925172/

 

[aruon]

Perception of what QA is like from anyone not involved or the profession or industry is almost universally off the mark. Here's a fairly good thread written by HoI4 QA on the matter:

https://forum.paradoxplaza.com/foru...-iron-iv-development-diary-55-hoi4-qa.925172/

ah, thanks jake.

 

[Bugnr01]

That was the reason why i asked.
Thx, totally forgot this DD.

 

[Seven Sulivin]

Actually, you know what would be neat? QA for historical accuracy.

 

[mathers]

Perception of what QA is like from anyone not involved in the profession or industry is almost universally off the mark. Here's a fairly good thread written by HoI4 QA on the matter:

https://forum.paradoxplaza.com/foru...-iron-iv-development-diary-55-hoi4-qa.925172/

This worries me. Looks like some junior end user type of persons both the embedded and central testers. That explains the regressions and imbalances we observe.

Hire proper software engineers into testing positions as well. Build a testlab. Do not use ui to test ai or balance... Build testability into Clausewitz.

 

[XYN]

This worries me. Looks like some junior end user type of persons both the embedded and central testers. That explains the regressions and imbalances we observe.

Hire proper software engineers into testing positions as well. Build a testlab. Do not use ui to test ai or balance... Build testability into Clausewitz.

As far as I know game development in general isn't that big on the automated testing.

But they probably have some, maybe a script which runs the game and emulates user actions to make sure that the basic stuff isn't broken, while these manual testers check it by hands.

 

[mathers]

As far as I know game development in general isn't that big on the automated testing.

But they probably have some, maybe a script which runs the game and emulates user actions to make sure that the basic stuff isn't broken, while these manual testers check it by hands.

Game development usually have 1-2 year support cycle and ui heavy.

Ckii is 5+ year, with 20+ update. And mayor complexity and most bug is not on ui layer. Calls for more conventional development methodology.