OpenCascade/occt
1
0
Fork 0
mirror of https://git.dev.opencascade.org/repos/occt.git synced 2025-08-14 13:30:48 +03:00
Code Activity

0024335: Draw Harness - support grids list in testgrid command

Browse Source 
Arguments of command testgrid specifying test group and grid to be executed now can be a list of file masks (space or comma separated).
One more optional argument added allowing to specify masks for test cases to be run.
This commit is contained in:
abv
2014-07-08 15:08:46 +04:00
committed by apn
parent a987c2f484
commit 87f42a3fb6
2 changed files with 41 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
dox/dev_guides/tests/tests.md
View File

@@ -56,7 +56,7 @@ return ;# this is to avoid an echo of the last command above in cout
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Note that variable *CSF_TestDataPath* is set to default value at DRAW start, pointing at the folder <i>$CASROOT/data</i>.
In this example, subdirectory <i>d:/occt/test-data</i> is added to this path. Similar code could be used on Linux and Mac OS X except that on non-Windows platforms colon ‘:’ should be used as path separator instead of semicolon ‘;’.
In this example, subdirectory <i>d:/occt/test-data</i> is added to this path. Similar code could be used on Linux and Mac OS X except that on non-Windows platforms colon ":" should be used as path separator instead of semicolon ";".
All tests are run from DRAW command prompt (run *draw.tcl* or *draw.sh* to start it).
@@ -70,12 +70,13 @@ Example:
Draw[]> testgrid
~~~~~
For running only a group or a grid of tests, give additional arguments indicating followed by path to the group and (if needed) the grid name.
For running only a subset of test cases, give masks for group, grid, and test case names to be executed.
Each argument is a list of comma- or space-separated file masks; by default "*" is assumed.
Example:
~~~~~
Draw[]> testgrid blend simple
Draw[]> testgrid bugs caf,moddata*,xde
~~~~~
@@ -106,7 +107,7 @@ If necessary, a non-default output directory can be specified using option <i>
Example:
~~~~~
Draw[]> testgrid –outdir d:/occt/last_results -overwrite
Draw[]> testgrid -outdir d:/occt/last_results -overwrite
~~~~~
In the output directory, a cumulative HTML report summary.html provides links to reports on each test case. An additional report in JUnit-style XML format can be output for use in Jenkins or other continuous integration system.
@@ -117,18 +118,20 @@ For example:
~~~~~
Draw[3]> help testgrid
testgrid: Run all tests, or specified group, or one grid
Use: testgrid [group [grid]] [options...]
Use: testgrid [groupmask [gridmask [casemask]]] [options...]
Allowed options are:
-parallel N: run N parallel processes (default is number of CPUs, 0 to disable)
-refresh N: save summary logs every N seconds (default 60, minimal 1, 0 to disable)
-outdir dirname: set log directory (should be empty or non-existing)
-overwrite: force writing logs in existing non-empty directory
-xml filename: write XML report for Jenkins (in JUnit-like format)
Groups, grids, and test cases to be executed can be specified by list of file
masks, separated by spaces or comma; default is all (*).
~~~~~
@subsubsection testmanual_1_3_3 Running a Single Test
To run a single test, type command *test*’ followed by names of group, grid, and test case.
To run a single test, type command *test* followed by names of group, grid, and test case.
Example:
@@ -147,7 +150,7 @@ To see intermediate commands and their output during the test execution, add one
The detailed rules of creation of new tests are given in <a href="#testmanual_3">section 3</a>. The following short description covers the most typical situations:
Use prefix “bug” followed by Mantis issue ID and, if necessary, additional suffixes, for naming the test script and DRAW commands specific for this test case.
Use prefix "bug" followed by Mantis issue ID and, if necessary, additional suffixes, for naming the test script and DRAW commands specific for this test case.
1. If the test requires C++ code, add it as new DRAW command(s) in one of files in *QABugs* package. Note that this package defines macros *QVERIFY* and *QCOMPARE*, thus code created for QTest or GoogleTest frameworks can be used with minimal modifications.
2. Add script(s) for the test case in grid (subfolder) corresponding to the relevant OCCT module of the group bugs <i>($CASROOT/tests/bugs)</i>. See <a href="#testmanual_5_2">the correspondence map</a>.
@@ -156,7 +159,7 @@ Use prefix “bug” followed by Mantis issue ID and, if necessary, additional s
* Use command *locate_data_file* to get a path to data files used by test script. (Make sure to have this command not inside catch statement if it is used.)
* Use DRAW commands to reproduce the situation being tested.
* If test case is added to describe existing problem and the fix is not available, add TODO message for each error to mark it as known problem. The TODO statements must be specific so as to match the actually generated messages but not all similar errors.
* Make sure that in case of failure the test produces message containing word “Error” or other recognized by test system as error (see files parse.rules).
* Make sure that in case of failure the test produces message containing word "Error" or other recognized by test system as error (see files parse.rules).
4. If the test case uses data file(s) not yet present in the test database, these can be put to subfolder data of the test grid, and integrated to Git along with the test case.
5. Check that the test case runs as expected (test for fix: OK with the fix, FAILED without the fix; test for existing problem: BAD), and integrate to Git branch created for the issue.
@@ -617,7 +620,8 @@ return ;# this is to avoid an echo of the last command above in cout
For better efficiency, on computers with multiple CPUs the tests can be run in parallel mode. This is default behavior for command *testgrid* : the tests are executed in parallel processes (their number is equal to the number of CPUs available on the system). In order to change this behavior, use option parallel followed by the number of processes to be used (1 or 0 to run sequentially).
Note that the parallel execution is only possible if Tcl extension package *Thread* is installed. It is included in *ActiveTcl* package, but can be absent in some Linux distributions. If this package is not available, *testgrid* command will output a warning message.
Note that the parallel execution is only possible if Tcl extension package *Thread* is installed.
If this package is not available, *testgrid* command will output a warning message.
@subsection testmanual_4_4 Checking non-regression of performance, memory, and visualization

50
src/DrawResources/TestCommands.tcl
View File

@@ -58,13 +58,15 @@ proc test {group grid casename {echo 0}} {
# Basic command to run indicated test case in DRAW
help testgrid {
Run all tests, or specified group, or one grid
Use: testgrid [group [grid]] [options...]
Use: testgrid [groupmask [gridmask [casemask]]] [options...]
Allowed options are:
-parallel N: run N parallel processes (default is number of CPUs, 0 to disable)
-refresh N: save summary logs every N seconds (default 600, minimal 1, 0 to disable)
-outdir dirname: set log directory (should be empty or non-existing)
-overwrite: force writing logs in existing non-empty directory
-xml filename: write XML report for Jenkins (in JUnit-like format)
Groups, grids, and test cases to be executed can be specified by list of file
masks, separated by spaces or comma; default is all (*).
}
proc testgrid {args} {
global env tcl_platform _tests_verbose
@@ -145,12 +147,14 @@ proc testgrid {args} {
}
# treat arguments not recognized as options as group and grid names
if { ! [info exists groupname] } {
set groupname $arg
} elseif { ! [info exists gridname] } {
set gridname $arg
if { ! [info exists groupmask] } {
set groupmask [split $arg ,]
} elseif { ! [info exists gridmask] } {
set gridmask [split $arg ,]
} elseif { ! [info exists casemask] } {
set casemask [split $arg ,]
} else {
error "Error: cannot interpret argument $narg ($arg): both group and grid names are already defined by previous args!"
error "Error: cannot interpret argument $narg ($arg)"
}
}
@@ -174,6 +178,11 @@ proc testgrid {args} {
error "Error: Cannot create directory \"$logdir\", or it is not writable"
}
# masks for search of test groups, grids, and cases
if { ! [info exists groupmask] } { set groupmask * }
if { ! [info exists gridmask ] } { set gridmask * }
if { ! [info exists casemask ] } { set casemask * }
######################################################
# prepare list of tests to be performed
######################################################
@@ -200,18 +209,8 @@ proc testgrid {args} {
continue
}
# if test group is specified, check that directory with given name exists in this dir
# if not, continue to the next test dir
if { [info exists groupname] && $groupname != "" } {
if { [file isdirectory $dir/$groupname] } {
set groups $groupname
} else {
continue
}
} else {
# else search all directories in the current dir
if [catch {glob -directory $dir -tail -types d *} groups] { continue }
}
# search all directories in the current dir with specified mask
if [catch {glob -directory $dir -tail -types d {*}$groupmask} groups] { continue }
# iterate by groups
if { $_tests_verbose > 0 } { _log_and_puts log "Groups to be executed: $groups" }
@@ -241,11 +240,13 @@ proc testgrid {args} {
continue
}
# if specific grid is requested, check that it is present; otherwise make complete list
if { ! [info exists gridname] || $gridname == "" || $gridname == $gridid || $gridname == $grid } {
# check that grid fits into the specified mask
foreach mask $gridmask {
if { $mask == $gridid || [string match $mask $grid] } {
lappend gridlist $grid
}
}
}
close $fd
# iterate by all grids
@@ -271,11 +272,14 @@ proc testgrid {args} {
if { $logdir != "" } { file mkdir $logdir/$group/$grid }
# iterate by all tests in the grid directory
if { [catch {glob -directory $griddir -type f *} testfiles] } { continue }
if { [catch {glob -directory $griddir -type f {*}$casemask} testfiles] } { continue }
foreach casefile [lsort -dictionary $testfiles] {
# filter out begin and end files
# filter out files with reserved names
set casename [file tail $casefile]
if { $casename == "begin" || $casename == "end" } { continue }
if { $casename == "begin" || $casename == "end" ||
$casename == "parse.rules" } {
continue
}
lappend tests_list [list $dir $group $grid $casename $casefile]
}