How is everyone approaching setting content budgets with regards to loading times?
Seems reasonable to run some initial tests to see number of MB you can load on hardware in a certain amount of time, and stress tests to try loading like 10k small files at once, and different types of data to get a feel for what you can reasonably do, and set initial guidelines based around those numbers.
Has anyone approached this differently?
@khalladay Had the same approach. Also worked backward from the data of a typical level (or similar time coherent data set).
"Too long: improve, reload, rince, repeat".
On medium to large project, you will go over budget.. One very good thing: log load times/package size/budget left for each loading (elastic search/kibana) and monitor every big step (bug? new data shipped?). Involve everyone in the monitoring (screen on the floor).
This might not apply to your prod but hey ^^
@jnq Getting everyone involved in the monitoring (and diagnosing / fixing issues) is my biggest concern.
Did you run into any issues when you started doing the monitoring from a team dynamic perspective? I'm trying to avoid it feeling like something that's making everyone's life more difficult.
@khalladay I really think that shipping the game is everyone's matter. The key element is to be very clear on when it's ok to go over budget and when it's not. The idea here is to identify singular anomalies early on. If you have a good cohesive team, people will get together and tackle the issues ("DT: Hey animators, data managers, devs, we have an issue with memory, how should we fix this"). If everyone knows it's fix time, your team dynamic should not shift (and that's what producers are for)
@khalladay Didn't have a very well defined process on my last game. We basically just ran the game on a min spec machine with a slow 5400 RPM HDD. If it felt too slow we profiled and looked for ways to bring it down. We had a loose goal of <10 seconds per level, I think. On recommended spec it was 2-3 seconds.
@abyrd This is good advice - we will definitely need to get our automated tests running on older handsets (since I'm in mobile land, my 5400 RPM is an iPhone 5 XD).
My main concern is how to get these sorts of tests integrated with team process in such a way that when a test breaks, it's something the whole team knows about, and feels like they have the tools to fix.
@khalladay Calibration as you describe is super important, even on "familiar" platforms. But raw IO is not always the biggest bottleneck - if you do any load-time processing be ready to stress test and profile it heavily. (Obviously preprocessing is gold where possible, but e.g. procedural gen can disrupt this.)
Mostly just be willing to adjust budgets during development, and watch regressions like a hawk. Automated tests are huge here, and graphs are awesome.