forked from firewave/mame_regtest
-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathTODOS
381 lines (363 loc) · 18.7 KB
/
TODOS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
mame_regtest
features:
- per-cart configuration
- check every game with created files (re-run after clean run)
- add option for testing of .inp file writing/reading (-record/-playback)
- add "start_at" for devices
- use transparent crosshair?
- MAME needs option to disable crosshair
- add "hang"/deadlock detection (-watchdog / usable from 0.136)
- multipass XPath (e.g. first select m6502 cpus and then all with autosave from those nodes)
- use '&' instead of '|'
- multithreading mode
- test thread code on linux
- use pthread on windows?
- how to handle chdir() stuff now?
- use fork code and set in forked code
- configuration rewrite
- empty options will not properly overwrite/reset existing options to default values
- print config values in read function (or put in array and print function for array)
- cleanup post-loading code of configuration (checks, output etc.)
- proper handling of default values in config (add additional entry in struct)
- move variables into the structure (only one dynamic string type)
- read all options as const to avoid modifications outside the config
- report unknown options
- use register function to add config values?
- add macro to output files/folders for version/revision etc.
- use the following debugger commands
- gtime <milliseconds in hex>
- hardreset (causes endless restarts of MAME)
- snap
- add option to override debugscript
- add soft/hardreset tests after each call like "autosave" test?
- how will it work with -autosave?
- output data enhancements
- add clean/merged mame_regtest configuration to output data
- add memory dumps to the output files for -debug runs?
- output the software list only once and only when used
- check for existing files
- use special prefix for non-output XMLs like config, softwarelists, etc. (maybe '_'?)
- mark the different autosave calls (load / save)
- store both output file states in case of autosave
- make it possible to set affinity of a run (SetProcessAffinityMask() / sched_setaffinity())
- add signal/abort handler and cleanup in it
- take a look at messtest for "hardreset" tests and extended image testing
- image testing
- attribute to disable an image entry in the XML
- add base folder for images in XML (could also be used for parsing a folder?)
- read carts from a directory?
- add parse flag?
- only use on relative path entries (x:\, /, \\, etc.)
- use folder node in-between to be able to switch folders
- read images from folder with same name as device (look for XML first / specify folder in XML?)
- add option with folder for device XMLs?
- add XPath support for software lists
- measure the real time necessary for the execution to detect slowdowns
- all bios/ramsize/configuration/dipswitch combinations
- call function with array
- process the first entry and call itself with a smaller array
- try to create append_to_array() based on split_string()
- handle multiple software lists in hack
- log info about softwarelist entries without matching interface
- how to specify a specific device in softwarelist tests?
- document all the exitcodes in README
- avoid duplicated cleanup code for dips and configurations - use generic function
- make pointer array dynamic
- test again with LOG_ALLOC and *all* features enabled
- prepare softwarelists before processing the data and store the path in the driver_info
- add append flag to copy_file()
- run without XML creation
- additional mode for "no_execution"
- add option "devicefile_dir" to load the mrt_*.xml from
- allow image paths to be absolute or relative to the current folder (no ..)
- change the image xml format to include driver subnodes, so you don't need one for each driver
- additional format? different filename than the others?
- add function to config.c/.h to store configuration as XML
- add support for multiple configurations
- mame_regtest <config1> <config2>
- change configuration to store the values in the big array and not external variables
- get rid of test-specific static variables
- make it possible to use multiple configurations
- config_object {
void* xml;
void* root;
config_entries[]
}
- SetNamedPipeHandleState
- PIPE_NOWAIT
- properly handle and report -listxml failure
- make it easier to perform batch processing by allowing config overrides via "-<key> <value>" command-line option?
- use_gdb
- check if gdb exists when use_gdb is set to 1
- gdb --batch hangs on linux since it waits for key input (ubuntu 10.04 x64)
- prevent gdb stdout output in frontend and -listxml files
- gdb output is not being demangled
- add --run-command="set print demangle on" or "set demangle-style auto"
- add schemas to XMLs
- need to quote all usage of "output_path"
- mrt_system() sometimes hangs
- errors about devices are written to stderr - how to get them?
- also create XMLs for the frontend functions
- -verifyroms needs -rp
- -verifysamples needs sample path
- use -samplepath
- check if image files exist before trying to load them
- how to use one XML with multiple drivers - check parent - child relationship?
- mame_regtest takes a lot of CPU after running a command with a lot of output - improve reading of stdout? use move instead of read and write?
- create small wrapper application to move chdir() out of main application to parallize processing
chdir()
system()
chdir()
- device bioses
- speed up -listxml writing
- add generic support to run MAME in a different program instead of hardcoding it to gdb and valgrind
- add drmemory support
- add windbg support
- how to support "slot in slot"?
- add support to run a select list of softlist items
- create testset for mandatory devices, that don't have a softlist
- add option to disable "verifysoftware" on older versions
- add filename to PNG and MNG parsing errors
- how to get rid of empty line when -listsoftware is written?
- get rid of execute_mame() result? only used in one place
- how to use single entries from each softwarelists - not just the first?
- add an option how many entries from each softlist to process when "use_softwarelist" is set (remove use_softwarelist=2?)
- extract MNG frame and do a pngcmp
- scan existing result and only run sets/combinations, that caused an error
- do not proceed when software list could not be written
- change EOL in SVN to native
- write all -listsoftware entries on start-up so they are verified before the test begins
- abort the test when anything fails?
- avoid multiple writing of softwarelists by saving them under the internal name and load them by that?
- use XPath to more easily filter entries out of softlist
-
mame_regtest.c:2247:20: warning: declaration of âxmlBufâ shadows a global declaration [-Wshadow]
In file included from /usr/include/libxml2/libxml/parser.h:16:0,
from mame_regtest.c:46:
/usr/include/libxml2/libxml/tree.h:88:27: warning: shadowed declaration is here [-Wshadow]
- add core dump handling/enabling
- multi custom commands
- load -listsofware XML on start-up to check for schema errors
- lookup interface via slots names?
- make software fitering optional
- remove use_sound and use_throttle
- add switch to test softlist entries with and without interface
- also run a softlist entry with empty slot with use_softwarelist=3
- make running with all parts another mode as well
- split up the "use_softwarelist" a bit - it's getting confusing
- group execution by sourcefile
- magictg.xml
Entity: line 3: parser error : invalid character in attribute value
0;Unhandled register: 3fff Unhandled register: 3fff 1c00 PFLAGS: ea
^
Entity: line 3: parser error : attributes construct error
0;Unhandled register: 3fff Unhandled register: 3fff 1c00 PFLAGS: ea
^
Entity: line 3: parser error : Couldn't find end of Start Tag result line 3
0;Unhandled register: 3fff Unhandled register: 3fff 1c00 PFLAGS: ea
^
Entity: line 3: parser error : PCDATA invalid Char value 12
0;Unhandled register: 3fff Unhandled register: 3fff 1c00 PFLAGS: ea
^
Entity: line 13: parser error : Opening and ending tag mismatch: output line 2 a
nd result
</result>
^
Entity: line 14: parser error : Extra content at the end of the document
</output>
^
- try to store timing information as well
- ALSA lib seq_hw.c:457:(snd_seq_hw_open) open /dev/snd/seq failed: Permission denied
sudo modprobe snd_seq
-
<result exitcode="0" stdout="WARNING: font Liberation Sans, is not TrueType or BDF, using MAME default 250 INITIALIZING...Average speed: 763.79% (1 seconds) 2560 tagmap lookups " stderr="ALSA lib confmisc.c:768:(parse_card) cannot find card '0' ALSA lib conf.c:4248:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings ALSA lib conf.c:4248:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory ALSA lib confmisc.c:1251:(snd_func_refer) error evaluating name ALSA lib conf.c:4248:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory ALSA lib conf.c:4727:(snd_config_expand) Evaluate error: No such file or directory ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM default ">
molecula.xml
Entity: line 3: parser error : invalid character in attribute value
^
Entity: line 3: parser error : attributes construct error
^
Entity: line 3: parser error : Couldn't find end of Start Tag result line 3
^
Entity: line 3: parser error : PCDATA invalid Char value 12
^
Entity: line 13: parser error : Opening and ending tag mismatch: output line 2 and result
</result>
^
Entity: line 14: parser error : Extra content at the end of the document
</output>
^
- review the default osdprocessors handling
- add "validate_only" option
- part not in wiki report
- error code 4 has no impact in statistics
- add "quiet" mode to report generation
- do not just check the interface on which device slot to run the sotware list entry, but also the extension (see aim65)
- add option to skip unsupported softlist entries
- use -debugger internal
- add a flag to the debugger to not break at all (prevent internal breakpoint hangs)
- store timing information and generate reports on how much time in a testrun is spent on which drivers/sets
- generate a report in the format I would send to the list
- add option to use grouped execution
- test differently sized software entries?
- do not show stdout in report entries, that don't need it (e.g. fatalerror)
- merge "use_softwarelist" modes 3 and 4?
- "ignore_exitcode_4" no longer working?
- always call softlist entries with softlist name prefix
- process entries from each softlist
- make the compatible softlist handling a hack for older versions?
- ignore nodump roms in software lists - amiga_workbench
- add wav/avi/png/mng valiation
- ffmpeg -v error -i file.avi -f null - 2>error.log
- avconv
- add -listnetwork
- add sanity checks before adding attributes to avoid invalid XMLs being created
- remove memory leak checking code in favor of LeakSanitizer?
internal:
- call parse_directory() in execute_mame() / call clear_directory() in execute_mame2()
- make pointers const (or const * const)
- move MNG frame parsing into seperate function
- add printf-format support to append_string()
- use functions for lists
- list_create()
- list_next_entry()
- list_insert() - sorted
- list_append() - unsorted
- list_free()
- rename CFG_STR_PTR to CFG_STR
- handle all options as string (use atoi() to convert to integers)
- factor out duplicated PNG/MNG code
- create open_mng_and_skip_sig() for PNG
- get MEND reference out of IDAT function
- change ram, bioses and devices from arrays to lists
- add sanity checks for maximum bios, ramsize and device count
- use different prefix for cfg_type enum / conflicts with config_entry
- run test_createconfig in dummy_root to avoid usage of existing mame.ini (and other other command-line option calls)
- use macros FREE(), XMLFREE() etc. which includes the setting to NULL
- port mrt_system() to non-windows systems
- replace the counts in the structures with NULL check instead
- fix warnings
- fix LOG_ALLOC
- get fully working with Visual Studio and profile it
issues/bugs/errorhandling:
- avoid duplicated slashes
- bail out when invalid config_entry_type found?
- test chdir() on start-up and bail out when it fails
- do not test dipswitch/configurations when additional information is missing in -listxml
- test with path with whitespaces
- are there dipswitches, that are depending on multiple ports?
- make "hack_pinmame" fully working
- add errorhandling to all libxml2 functions
- bail out when any of the postfix counters would overflow
documentation:
- wiki
- the different builds
- how the results are created
- the machine the tests are run on
- the resulting command-line
- a run on the official binary once the cycle is completed
- runs with the usual builds on the u-releases
- add MAME version to hack_mngwrite description (change seems to be in 0.108u5)
- add notes about dependencies between options in the documentation
- add note to "use_debug" about needing a build with USE_DEBUGGER=1 and about the version, that started to include the debugger in non-debug builds
- update building instructions in Wiki
- verify the version number for software lists
- add link to configuration on main mame_regtest page
- document upper limit of dipswitches/devices/etc. of 100000
- put all the wiki documentation up on breaken.de
compilation:
- fix compilation on systems, that already have a htonl() function
- #undef htonl might help
- change LITTLE_ENDIAN/BIG_ENDIAN defines - they are both defined on some systems (maybe BYTE_ORDER? LSB_FIRST?)
- make doesn't work when the additional files already exist and you compile by target (e.g. mame_regtest.exe)
Visual Studio:
- move the Microsoft-specific defines into a seperate header
create_report:
- write comparison tool - FIXED?
- parse dir A and check dir B for missing file
- parse dir B and check dir A for new files
- show the following
- files differ (PNG/MNG frame diffs)
- exitcode differs
- files missing
- new files
- stdout/stderr differs
- add documentation for configuration
- HTML report
- create HTML report (different variants)
Unexpected Returnvalues (sourcefile - driver - returncode - details [flags, stdout])
Untested Games (sourcefile - driver)
Image Comparison results (sourcefile - driver / with images links - before and after)
All Result Images
- add paging to HTML report (by letters / x entries per page)
- add all messages to HTML report
- add all options to HTML report
- put HTML report into a folder and copy pictures in a <html_report_name>_new/<html_report_name>_ref folder
- add option for main HTML report folder
- make HTML report work in Mozilla ( use /ref and /new links)
- extract differing frames (also for HTML report)
- show PNG differences (use pngcmp) - FIXED?
- expose the data (collect the necessary XMLs and add them to a subfolder?)
- wiki syntax
| row1 | row 2 |
- create an intermediate format to process, so it's not necessary to process all output again when a new version comes out
- add option to merge the output to a report
- properly handle sets with autosave (use index in XPath to access the second set of nodes when both support autosave)
- handle missing attributes properly
- add the various XPaths as options in the configuration
- create default configuration for create_report to match removed options
- result_file
- add some useful reports from old create_report
- use proper formating for "(autosave)" string
- use grep to diff each file
- regex for auto.sta "^[+-].*auto.sta"
- check for "cheat expression" errors in stderr
- add check for a specific string as option
- replace "called within reset scope by" with it -> field="stderr" string="called within reset scope by"
- allow multiple ones
- replace hard-coded format with templates
- use tables in comparison report?
- create output_folder
- same behavior as mame_regtest output folder
- convert \n to <br> in stdout and stderr output
- how to support multiples paths for comparison reports?
- try to detect new/missing sets in comparison separately
- note differences about bios or ramsize etc. in results
- when merged configuration exists in results folder add information to report
- comparison doesn't create output folder - doesn't complain when it's missing either
- add optional to list OPTIONAL warnings
- add option to print negative exitcodes in hex
- add reports, that parse -log output for like "unmapped writes" and stuff
- add HTML version of the error report
- filter gdb output
- stdout
- "[New Thread"
- "[Inferior"
- stderr
- "No stack."
- in comparison report show which is which
- parse the valgrind.log for
- "uninit"
- "Invalid"
- "Source and destination overlap"
- add option to create_report to bail out on XML error
- check print_stderr - seems to include all drivers in report
- how to get a sorted filelist on linux?
- speed report
gp2x.c: gp2x - Average speed: 0.00%
hyperscan.c: hs - Average speed: 0.00%
octopus.c: octopus - Average speed: 0.00%
- make the pngcmp path configurable so the file doesn't need to be copied
- add option to ignore missing roms in report
create_image_xml
- add more output to create_image_xml
pngcmp:
- how to speed up writing of diff?
- try to optimize it, but comparing and writing in the same loop and just don't write the final bitmap when they differ
- package optimized mingw binary instead of Visual Studio one
Known Issues:
- not everything is properly encoded by xmlEncodeEntitiesReentrant()
- using autosave will overwrite the initial valgrind log-file for the driver (well..not really, it adds the pid - still you want to differ them easily)
Misc:
- create patch to write ui_popup() to -log output
- MAME patches
- add "exit"/"run-and-hide" command to debugger