From: Eduardo Ramos Testillano Date: Sat, 11 Apr 2020 23:49:21 +0000 (+0200) Subject: Improvements from anna fork X-Git-Url: https://git.teslayout.com/public/public/public/?p=anna.git;a=commitdiff_plain;h=2e2f6d4e2ffe1c8b86c812807f0e501ab78f56f9 Improvements from anna fork * Rename steps that are diameter-specific Diameter steps should be renamed to clarify the real context of use. * Change ADML point of generation for launcher.traces Change from the physical executable location to the execution point (current working directory) * Improve final detection algorithm Improves resolution to detect the end of the testing. Uses the number of finished test cases (success + failed) instead the initialized ones (which is less safer because initial conditions). * Add dynamic ip-limit control Add the possibility to control de ip-limit from test case by mean a new generic action: ip_limit * Manage terminate signal from ADML application * Improve counters Implement signalTermination handler and force counter records. If counters dumps are disabled (--cntRecordPeriod 0) the file won't be created anyway. Also, improve the mask for counters file creation instead of modifying it on close (avoid possible stop problems). We set 644 for counters. * Implement junit report generation Add new methods and implement new asXML getter for junit report. * AOTS (Agents-Oriented Testing Setup): evolution from original idea of diameter agents but extended to any kind of resource. Change-Id: I4dc3ac41b86709439ffa2aa98aa011d956d8df2f --- diff --git a/INSTALL_AOTS.md b/INSTALL_AOTS.md new file mode 100644 index 0000000..983b1e6 --- /dev/null +++ b/INSTALL_AOTS.md @@ -0,0 +1,10 @@ +# Agents-Oriented Testing Setup (AOTS) + +## Deployment + +Execute 'example/diameter/launcher/deploy-aots.sh' and follow instructions. + +## Documentation + +See README.md file at installation target directory. + diff --git a/README.md b/README.md index 0c0cfd4..5bf77ea 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,7 @@ You could specify type of build, 'Debug' or 'Release', for example: ### OpenSSL: > sudo apt-get install libssl-dev - + ### Gnome XML: > sudo apt-get install libxml2-dev @@ -83,7 +83,7 @@ You could specify type of build, 'Debug' or 'Release', for example: > cd /usr/src/gtest > sudo cmake CMakeLists.txt > sudo make - + Copy or symlink libgtest.a and libgtest_main.a to your /usr/lib folder: > sudo cp *.a /usr/lib @@ -120,3 +120,7 @@ Optionally you could specify another prefix for installation: > cat install_manifest.txt | sudo xargs rm +## AOTS (Agents-Oriented Testing Setup) + +See INSTALL_AOTS.md + diff --git a/dynamic/launcher/gx/00001/Procedure.cpp b/dynamic/launcher/gx/00001/Procedure.cpp index 20af42c..99a87fe 100644 --- a/dynamic/launcher/gx/00001/Procedure.cpp +++ b/dynamic/launcher/gx/00001/Procedure.cpp @@ -262,19 +262,19 @@ void Procedure::execute(const std::string &args, std::string &response) throw(a } // Step 2: sendxml2e: CCR-Initial - tc->addSendxml2e(ccri.code(), originHost, -1 /* 'wait for request' step number for answers */); + tc->addSendDiameterXml2e(ccri.code(), originHost, -1 /* 'wait for request' step number for answers */); // Step 3: waitfe: CCA with same session id // PARAM: 1 2 3 4 5 6 7 8 9 10 11 // wait|[code]|[bitR]|[hopByHop]|[applicationId]|[sessionId]|[resultCode]|[msisdn]|[imsi]|[serviceContextId] - tc->addWait(true /* from entity */, "272", "0", "", "", sessionId, "2001", "", "", ""); + tc->addWaitDiameter(true /* from entity */, "272", "0", "", "", sessionId, "2001", "", "", ""); if (haveTermination) { // Step 4: sendxml2e: CCR-Termination - tc->addSendxml2e(ccrt.code(), originHost, -1 /* 'wait for request' step number for answers */); + tc->addSendDiameterXml2e(ccrt.code(), originHost, -1 /* 'wait for request' step number for answers */); // Step 5: waitfe: CCA with same session id - tc->addWait(true /* from entity */, "272", "0", "", "", sessionId, "2001", "", "", ""); + tc->addWaitDiameter(true /* from entity */, "272", "0", "", "", sessionId, "2001", "", "", ""); } } // loop diff --git a/example/diameter/launcher/Launcher.cpp b/example/diameter/launcher/Launcher.cpp index 94d8f66..2175263 100644 --- a/example/diameter/launcher/Launcher.cpp +++ b/example/diameter/launcher/Launcher.cpp @@ -867,6 +867,15 @@ void Launcher::resetCounters() throw() { anna::diameter::codec::OamModule::instantiate().resetCounters(); } +void Launcher::signalTerminate() throw(anna::RuntimeException) { + LOGMETHOD(anna::TraceMethod tm("Launcher", "signalTerminate", ANNA_FILE_LOCATION)); + + forceCountersRecord(); + + a_communicator->terminate (); + comm::Application::signalTerminate (); +} + void Launcher::signalUSR2() throw(anna::RuntimeException) { std::string inputFile = getSignalUSR2InputFile(); @@ -1220,7 +1229,17 @@ std::string Launcher::help() const throw() { result += "\n values (Session-Id, Subscriber-Id, etc.)."; result += "\n"; result += "\n : commands to be executed for the test id provided. Each command programmed"; - result += "\n constitutes a test case 'step', numbered from 1 to N."; + result += "\n constitutes a test case 'step', numbered from 1 to N, with an exception for"; + result += "\n 'description' which is used to describe the test case:"; + result += "\n"; + result += "\n description| Sets a test case description. Test cases by default are"; + result += "\n constructed with description 'Testcase_'."; + result += "\n"; + result += "\n ip-limit[|amount] In-progress limit of test cases controlled from this test."; + result += "\n No new test cases will be launched over this value (test"; + result += "\n manager tick work will be ignored). Zero-value is equivalent"; + result += "\n to stop the clock tick, -1 is used to specify 'no limit' which"; + result += "\n is the default. For missing amount, value of 1 is applied."; result += "\n"; result += "\n timeout| Sets an asynchronous timer to restrict the maximum timeout"; result += "\n until last test step. Normally, this command is invoked"; @@ -1433,10 +1452,10 @@ std::string Launcher::help() const throw() { result += "\n next programmed test cases (1 by default). This event works regardless the timer tick"; result += "\n function, but it is normally used with the test manager tick stopped."; result += "\n"; - result += "\n test|ip-limit[|amount] In-progress limit of test cases. No new test cases will be launched over this value"; - result += "\n (test Manager tick work will be ignored). Zero-value is equivalent to stop the clock."; - result += "\n tick, -1 is used to specify 'no limit' which is the default. If missing amount, the"; - result += "\n limit and current amount of in-progress test cases will be shown."; + result += "\n test|ip-limit[|amount] In-progress limit of test cases established from global context. This value will be"; + result += "\n overwritten if used at test case level when the corresponding test step is executed."; + result += "\n Anyway, the meaning is the same in both contexts. But now, when the amount is missing,"; + result += "\n the limit and current amount of in-progress test cases will be shown."; result += "\n"; result += "\n test|goto| Updates current test pointer position."; result += "\n"; @@ -1444,13 +1463,14 @@ std::string Launcher::help() const throw() { result += "\n Test cases reports are not dumped on process context (too many information in general)."; result += "\n The report contains context information in every moment: this operation acts as a snapshot."; result += "\n"; - result += "\n test|interact|amount|id Makes interactive a specific test case id. The amount is the margin of execution steps"; - result += "\n to be done. Normally, we will execute 'test|interact|0|', which means that"; + result += "\n test|interact|amount[|id] Makes interactive a specific test case id. The amount is the margin of execution steps"; + result += "\n to be done. Normally, we will execute 'test|interact|0[|]', which means that"; result += "\n the test case is selected to be interactive, but no step is executed. Then you have to"; result += "\n interact with positive amounts (usually 1), executing the provided number of steps if"; result += "\n they are ready and fulfill the needed conditions. The value of 0, implies no execution"; result += "\n steps margin, which could be useful to 'freeze' a test in the middle of its execution."; result += "\n You could also provide -1 to make it non-interactive resuming it from the current step."; + result += "\n By default, current test case id is selected for interaction."; result += "\n"; result += "\n test|reset|<[soft]/hard>[|id] Reset the test case for id provided, all the tests when missing. It could be hard/soft:"; result += "\n - hard: you probably may need to stop the load rate before. This operation initializes"; @@ -1468,8 +1488,19 @@ std::string Launcher::help() const throw() { result += "\n been configured for the test case, hard reset could prevent stuck on the next cycle for"; result += "\n those test cases still in progress."; result += "\n"; + result += "\n test|initialized Shows the number of initialized test cases. Zero-value means that everything was processed"; + result += " or not initiated yet.\n"; + result += "\n"; + result += "\n test|finished Shows the number of finished (successful or failed) test cases."; + result += "\n"; result += "\n test|clear Clears all the programmed test cases and stop testing (if in progress)."; result += "\n"; + result += "\n test|junit Shows the junit report in the moment of execution."; + result += "\n"; + result += "\n test|summary-counts Test manager counts report. Counts by state and prints total verdict."; + result += "\n"; + result += "\n test|summary-states Test manager states report."; + result += "\n"; result += "\n test|summary Test manager general report (number of test cases, counts by state, global configuration,"; result += "\n forced in-progress limitation, reports visibility, etc.). Be careful when you have reports"; result += "\n enabled because the programmed test cases dumps could be heavy (anyway you could enable the"; @@ -1492,6 +1523,8 @@ std::string Launcher::help() const throw() { result += "\n"; result += "\n test|report-hex[|[yes]|no] Reports could include the diameter messages in hexadecimal format. Disabled by default."; result += "\n"; + result += "\n test|dump-stdout[|[yes]|no] Test manager information is dumped into stdout."; + result += "\n"; result += "\n"; result += "\n------------------------------------------------------------------------------------- Dynamic procedure"; result += "\n"; @@ -2094,6 +2127,14 @@ void Launcher::eventOperation(const std::string &operation, std::string &respons testManager.setDumpHex((param2 == "yes")); opt_response_content += (testManager.getDumpHex() ? "report includes hexadecimal messages" : "report excludes hexadecimal messages"); } + else if(param1 == "dump-stdout") { + if (numParams > 2) + throw anna::RuntimeException("Wrong body content format on HTTP Request. Use 'help' management command to see more information.", ANNA_FILE_LOCATION); + + if(param2 == "") param2 = "yes"; + testManager.setDumpStdout((param2 == "yes")); + opt_response_content += (testManager.getDumpHex() ? "test manager dumps progress into stdout" : "test manager does not dump progress into stdout"); + } else if(param1 == "goto") { if (numParams > 2) throw anna::RuntimeException("Wrong body content format on HTTP Request. Use 'help' management command to see more information.", ANNA_FILE_LOCATION); @@ -2132,14 +2173,14 @@ void Launcher::eventOperation(const std::string &operation, std::string &respons } } else if (param1 == "interact") { - if (numParams != 3) + if (numParams != 2) throw anna::RuntimeException("Wrong body content format on HTTP Request. Use 'help' management command to see more information.", ANNA_FILE_LOCATION); int amount = atoi(param2.c_str()); if (amount < -1) throw anna::RuntimeException("Interactive amount must be -1 (to disable interactive mode) or a positive number.", ANNA_FILE_LOCATION); - int id = atoi(param3.c_str()); + int id = ((param3 != "") ? atoi(param3.c_str()) : -1); anna::testing::TestCase *testCase = testManager.findTestCase(id); if (testCase) { if (amount == -1) { @@ -2203,6 +2244,18 @@ void Launcher::eventOperation(const std::string &operation, std::string &respons testManager.setAutoResetHard(param2 == "hard"); opt_response_content += anna::functions::asString("Auto-reset configured to '%s'", param2.c_str()); } + else if(param1 == "initialized") { + if (numParams > 1) + throw anna::RuntimeException("Wrong body content format on HTTP Request. Use 'help' management command to see more information.", ANNA_FILE_LOCATION); + + opt_response_content = anna::functions::asString("%lu", testManager.getInitializedCount()); + } + else if(param1 == "finished") { + if (numParams > 1) + throw anna::RuntimeException("Wrong body content format on HTTP Request. Use 'help' management command to see more information.", ANNA_FILE_LOCATION); + + opt_response_content = anna::functions::asString("%lu", testManager.getFinishedCount()); + } else if(param1 == "clear") { if (numParams > 1) throw anna::RuntimeException("Wrong body content format on HTTP Request. Use 'help' management command to see more information.", ANNA_FILE_LOCATION); @@ -2214,6 +2267,18 @@ void Launcher::eventOperation(const std::string &operation, std::string &respons opt_response_content = "there are not programmed test cases to be removed"; } } + else if(param1 == "junit") { + response_content = testManager.junitAsXMLString(); + return; + } + else if(param1 == "summary-counts") { + response_content = testManager.summaryCounts(); + return; + } + else if(param1 == "summary-states") { + response_content = testManager.summaryStates(); + return; + } else if(param1 == "summary") { response_content = testManager.asXMLString(); return; @@ -2225,6 +2290,8 @@ void Launcher::eventOperation(const std::string &operation, std::string &respons // PARAM: 1 2 3 4 5 6 7 8 9 10 11 // test|| + // description| + // ip-limit[|] // timeout| // sendxml2e| [|] // sendxml2c| [|] @@ -2236,7 +2303,19 @@ void Launcher::eventOperation(const std::string &operation, std::string &respons if(param2 == "") throw anna::RuntimeException("Missing command for test id operation", ANNA_FILE_LOCATION); // Commands: - if (param2 == "timeout") { + if (param2 == "description") { + if (numParams > 3) + throw anna::RuntimeException("Wrong body content format on HTTP Request. Use 'help' management command to see more information.", ANNA_FILE_LOCATION); + if(param3 == "") throw anna::RuntimeException("Missing description for test case", ANNA_FILE_LOCATION); + testManager.getTestCase(id)->setDescription(param3); // creates / reuses + } + else if (param2 == "ip-limit") { + if (numParams > 3) + throw anna::RuntimeException("Wrong body content format on HTTP Request. Use 'help' management command to see more information.", ANNA_FILE_LOCATION); + unsigned int limit = (param3 == "") ? 1 : atoi(param3.c_str()); + testManager.getTestCase(id)->addIpLimit(limit); // creates / reuses + } + else if (param2 == "timeout") { if (numParams > 3) throw anna::RuntimeException("Wrong body content format on HTTP Request. Use 'help' management command to see more information.", ANNA_FILE_LOCATION); if(param3 == "") throw anna::RuntimeException("Missing milliseconds for 'timeout' command in test id operation", ANNA_FILE_LOCATION); @@ -2257,9 +2336,9 @@ void Launcher::eventOperation(const std::string &operation, std::string &respons int stepNumber = ((param4 != "") ? atoi(param4.c_str()):-1); if (param2 == "sendxml2e") - testManager.getTestCase(id)->addSendxml2e(codecMsg.code(), getOperatedHost(), stepNumber); // creates / reuses + testManager.getTestCase(id)->addSendDiameterXml2e(codecMsg.code(), getOperatedHost(), stepNumber); // creates / reuses else - testManager.getTestCase(id)->addSendxml2c(codecMsg.code(), getOperatedHost(), stepNumber); // creates / reuses + testManager.getTestCase(id)->addSendDiameterXml2c(codecMsg.code(), getOperatedHost(), stepNumber); // creates / reuses } else if (param2 == "delay") { if (numParams > 3) @@ -2273,7 +2352,7 @@ void Launcher::eventOperation(const std::string &operation, std::string &respons throw anna::RuntimeException("Wrong body content format on HTTP Request. Use 'help' management command to see more information.", ANNA_FILE_LOCATION); if (param3 != "" || param4 != "" || param5 != "" || param6 != "" || param7 != "" || param8 != "" || param9 != "" || param10 != "" || param11 != "") { bool fromEntity = (param2.substr(4,2) == "fe"); - testManager.getTestCase(id)->addWait(fromEntity, param3, param4, param5, param6, param7, param8, param9, param10, param11); + testManager.getTestCase(id)->addWaitDiameter(fromEntity, param3, param4, param5, param6, param7, param8, param9, param10, param11); } else { throw anna::RuntimeException(anna::functions::asString("Missing condition for '%s' command in test id operation", param2.c_str()), ANNA_FILE_LOCATION); @@ -2302,7 +2381,7 @@ void Launcher::eventOperation(const std::string &operation, std::string &respons } bool fromEntity = (param2.substr(4,2) == "fe"); - testManager.getTestCase(id)->addWaitRegexpHex(fromEntity, regexp); + testManager.getTestCase(id)->addWaitDiameterRegexpHex(fromEntity, regexp); } else if ((param2 == "waitfe-xml")||(param2 == "waitfc-xml")) { if (numParams > 4) @@ -2362,7 +2441,7 @@ void Launcher::eventOperation(const std::string &operation, std::string &respons } bool fromEntity = (param2.substr(4,2) == "fe"); - testManager.getTestCase(id)->addWaitRegexpXml(fromEntity, regexp); + testManager.getTestCase(id)->addWaitDiameterRegexpXml(fromEntity, regexp); } else if (param2 == "sh-command") { // Allow pipes in command: diff --git a/example/diameter/launcher/Launcher.hpp b/example/diameter/launcher/Launcher.hpp index 1070e40..c97fe7b 100644 --- a/example/diameter/launcher/Launcher.hpp +++ b/example/diameter/launcher/Launcher.hpp @@ -113,6 +113,7 @@ public: void resetStatistics() throw(); void resetCounters() throw(); void signalUSR2() throw(anna::RuntimeException); + void signalTerminate() throw(anna::RuntimeException); std::string help() const throw(); anna::xml::Node* oamAsXML(anna::xml::Node* parent) const throw(); anna::xml::Node* statsAsXML(anna::xml::Node* parent) const throw(); diff --git a/example/diameter/launcher/MyCounterRecorder.cpp b/example/diameter/launcher/MyCounterRecorder.cpp index dd32652..fd29422 100644 --- a/example/diameter/launcher/MyCounterRecorder.cpp +++ b/example/diameter/launcher/MyCounterRecorder.cpp @@ -34,7 +34,7 @@ void MyCounterRecorder::open() throw(anna::RuntimeException) { anna::Logger::debug(msg, ANNA_FILE_LOCATION); ); - if((a_stream = ::open(a_fileName.c_str(), O_WRONLY | O_CREAT, S_IWUSR)) == -1) + if((a_stream = ::open(a_fileName.c_str(), O_WRONLY | O_CREAT, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH /*0644*/)) == -1) throw RuntimeException(anna::functions::asString("Error opening file '%s'; errno = %d", a_fileName.c_str(), errno), ANNA_FILE_LOCATION); sprintf(str, "%04d-%02d-%02d %02d:%02d|%04d-%02d-%02d %02d:%02d", @@ -61,7 +61,7 @@ void MyCounterRecorder::close() throw() { a_stream = -1; } -chmod(a_fileName.c_str(), S_IWUSR | S_IRUSR); +//chmod(a_fileName.c_str(), S_IWUSR | S_IRUSR); a_previousTime = ::time(NULL); } diff --git a/example/diameter/launcher/deploy-aots.sh b/example/diameter/launcher/deploy-aots.sh new file mode 100755 index 0000000..8a7f8b7 --- /dev/null +++ b/example/diameter/launcher/deploy-aots.sh @@ -0,0 +1,110 @@ +#!/bin/bash + +############# +# VARIABLES # +############# +SCR_DIR=`readlink -f $0 | xargs dirname` +PROJECT_ROOT=$(readlink -f $SCR_DIR/../../..) + +############# +# FUNCTIONS # +############# +_exit () { + local msg="$1" + local rc=$2 + [ -z "$rc" ] && rc=1 + + # Exit message: + [ -n "$msg" ] && echo -e "\n${msg}\n" + + exit $rc +} + +############# +# EXECUTION # +############# + +echo +echo "------------------------------------------------------" +echo " AGENTS-ORIENTED TESTING SETUP installation " +echo "------------------------------------------------------" +echo +[ -d $PROJECT_ROOT/build/Release ] && VARIANT=Release +[ -d $PROJECT_ROOT/build/Debug ] && VARIANT=Debug +[ -z "$VARIANT" ] && _exit "Cannot locate neither 'Release' nor 'Debug' variant !" +build_type_letter=$(echo $VARIANT | cut -c1 | tr '[:upper:]' '[:lower:]') + +version__dflt=v"`date +'%y'`.`date +'%m'`.`date +'%d'`$build_type_letter" +INSTALL_AOTS__dflt=$HOME/3rdParty/anna-aots-builds/v"`date +'%y'`.`date +'%m'`.`date +'%d'`$build_type_letter" + +echo "Choose the target path for installation [$INSTALL_AOTS__dflt]:" +echo " (enter a non-existent directory)" +read INSTALL_AOTS +[ -z "$INSTALL_AOTS" ] && INSTALL_AOTS=$INSTALL_AOTS__dflt +INSTALL_AOTS=`readlink -m $INSTALL_AOTS` +[ -d $INSTALL_AOTS ] && _exit "The target installation directory ($INSTALL_AOTS) already exists ! (if you want to reinstall, remove it first)" + +echo +echo "Stage 1: Deploying resources ......................" +echo +mkdir -p $INSTALL_AOTS +cp -r $SCR_DIR/deployments/aots/* $INSTALL_AOTS + +# VARIABLES ###################################### +# Sources: +LDIR=${PROJECT_ROOT}/example/diameter/launcher +BIN_DIR=${PROJECT_ROOT}/build/$VARIANT/bin +LIB_DIR=${PROJECT_ROOT}/build/$VARIANT/lib + +# Targets: +ADML=${INSTALL_AOTS}/agents/ADML +DTDs=${ADML}/DTDs +DYNLIBS=${ADML}/dynlibs +################################################## + +echo +echo "Variant: $VARIANT" +echo + +# Empty directories: +mkdir -p ${ADML}/counters +mkdir -p ${ADML}/test-reports + +# Scripts: +cp ${PROJECT_ROOT}/example/diameter/launcher/resources/scripts/operation_signal.sh ${ADML}/operation.sh + +# Templates: +mkdir $DTDs +cp ${PROJECT_ROOT}/include/anna/diameter/codec/message.dtd ${DTDs} +cp ${PROJECT_ROOT}/include/anna/diameter/stack/dictionary.dtd ${DTDs} +cp ${PROJECT_ROOT}/example/diameter/launcher/resources/services_examples/services.dtd ${DTDs} + +# Main Launcher and dynamic libraries: +cp ${BIN_DIR}/anna_diameter_launcher ${ADML}/ADML +cp -r $LIB_DIR/dynamic/launcher ${DYNLIBS} +cp $LIB_DIR/libanna_testing_shared.so ${DYNLIBS} + +# Get stuff from leaf directories: +cd $LIB_DIR/dynamic/launcher +leafs=( $(find . -type d -links 2) ) +cd - >/dev/null + +cd ${PROJECT_ROOT}/dynamic/launcher +for dir in ${leafs[@]} +do + cp $dir/*.xml ${DYNLIBS}/$dir 2>/dev/null + cp $dir/dynamic.suffix ${DYNLIBS}/$dir 2>/dev/null + cp -r $dir/services ${DYNLIBS}/$dir 2>/dev/null +done +cd - >/dev/null + +# ADML dynamic libs selection script: +cp ${LDIR}/resources/scripts/select_dynlib.sh ${DYNLIBS}/select.sh + +# Default dynamic library: +cd ${DYNLIBS} +ln -sf default/libanna_launcher_procedure_default_shared.so +cd - >/dev/null + +_exit "Done!" 0 + diff --git a/example/diameter/launcher/deployments/aots/README.md b/example/diameter/launcher/deployments/aots/README.md new file mode 100644 index 0000000..35570c1 --- /dev/null +++ b/example/diameter/launcher/deployments/aots/README.md @@ -0,0 +1,495 @@ +# Anna Agents-Oriented Testing Setup + +## Agents configuration + +``` +$> python3 loader.py -h +usage: loader.py [-h] -f FILE + +Anna Agents-Oriented Testing Setup Loader + +optional arguments: + -h, --help show this help message and exit + -f FILE, --file FILE Agents yaml configuration file +``` + +The python script 'loader.py' will preload and configure all the needed agents. +Also, ADML ones will start with the configured endpoints, ready for work. + +``` +python3 loader.py --file /agents.yml +``` + +See example at './tests.example/agents.yml'. + +It is recommended to have that file within the tests directory compatible with such layout. +Anyway, you could have a common agents definition for multiple directories wherever you want to store them. +Next procedure (execution) will get a working directory which will be the parent for all the tests processed. + +Also, you could load several agents definitions. + +The format for agents file specification is: + +For KAFKA: + +``` +: + description: + template: KAFKA + topic: +``` + +For ADML (diameter): + +``` +: + description: + template: ADML + application_id: + dictionary: + type: client | server + origin_host: + origin_realm: + address: +``` + +For HTTPMOCK: + +``` +: + description: + template: HTTPMOCK + address: +``` + +Example: + +``` +AF: + description: AF Diameter Rx Node + template: ADML + application_id: 16777236 + dictionary: stacks/DictionaryRx.16777236.xml + type: client + origin_host: machine.source.server.realm.com + origin_realm: source.server.realm.com + address: 127.0.0.1:3868 + +SMPC: + description: SMPC N7 Node + template: KAFKA + topic: n7topic + +DB: + description: database + template HTTPMOCK + address: dbnode:9090 +``` + + +## Test launcher + +``` +$> python3 launcher.py -h +usage: launcher.py [-h] -t TESTS_DIR [-k] [-s] [-i] [-d] [1/708] + +Anna Agents-Oriented Testing Setup Launcher + +optional arguments: + -h, --help show this help message and exit + -t TESTS_DIR, --tests-dir TESTS_DIR + Tests parent directory where to find .yml files (from + the next directories level) + -k, --keep-list-if-exists + Keeps intact the list of test cases (/launcher.list), creates it if missing + -s, --stop-adml-at-the-end + At the end, ADML keeps running to ease debugging. You + could force stop with this option + -i, --interactive Interactive execution to ease debugging of test cases + -d, --dry-run Used to test and debug provision, no execution is + launched + -p IP_LIMIT, --ip-limit IP_LIMIT + In-Progress limit is the number of coexisting In- + Progress State test cases. Defaults to 1 (sequential), + -1 would be 'no limit'). + -r TTPS, --ttps TTPS Rate of test cases launched (test ticks per second). + By default 50 (recommended for monothread version). +``` + +The python script 'launcher.py' will execute the test cases under the provided directory. +Test cases at the directory provided itself will be ignored (searchs are performed at second level in order to avoid catching agents definitions). +Then we recommend this structure: + +``` +/agents.yml +//.../.yml +//...//.../.yml +``` + +The organization of test cases, chapters, test suites, or whatever desired concept is completely free for user preferences. Anyway, we recommend the use of numeric prefixes to control the default order of execution (alphabetical order). + +To execute the tests, just provide the global parent directory. +NO TEST CASES SHALL BE LOCATED THERE BY CONVENTION (indeed, you could have them, but they would be ignored). + +``` +python3 launcher.py --tests-dir +``` + +The procedure will look for the file ```''/launcher.list'``` in order to plan the execution. +That list will be created/refreshed always, except if you pass the option '-k|--keep-list-if-exists' (in case you want to play with edited lists). +The algorithm to create the list is based in a recursive search of yaml files in alphabetical order under the tests directory provided, in a similar way than the output of: + +``` +find -mindepth 2 -name "*.yml" | sort -t'/' +``` + +### Interactive mode + +One interesting feature from AOTS is the possibility to interact test cases even to deep level (step by step). +You could enable the interactive mode by mean: + +``` +python3 launcher.py --tests-dir --interactive +``` + +A menu will be shown after test cases programming to give you a complete control of the ADML operations: + +``` + MAIN INTERACTIVE MENU + ===================== + (prefix option with 'h' to get detailed help) + + General: + 0. Exit/Quit + + Position: + 1. Go to + 2. Look + + Test cases execution: + 3. Start with test rate + 4. Start next N test cases + 5. In-progress limit + + Low level execution: test case steps + 6. Execute next N steps + + Status & cycling: + 7. Reset + 8. Pool repeats + 9. Collect + 10. Auto reset + 11. Reports configuration +``` + +### User-defined lists + +You could edit such list and remove or comment (#) entries (each line is a test case) in order to "disable" such test cases. +You could also mess the order with something like: + +``` +sort -R /launcher.list +``` + +Don't forget to pass the option '-k|--keep-list-if-exists' or the list will be overwritten during launcher procedure execution. + +### Test cases + +Files used by parsers within the test cases (xml diameter messages, json kafka ones, etc.), are relative to the test case file path location, anyway you could set absolute paths for the stuff used. + +Recomendations: + +* Alternative 1: create a directory called ```'$(basename .yml)'``` + in order to keep order: + +``` + /tc1-initial-with-ipv4.yml + /tc1-initial-with-ipv4/AAA.xml + /tc1-initial-with-ipv4/smpc-consumed.json + /tc2-initial-with-ipv6.yml + /tc2-initial-with-ipv6/AAA.xml + /tc2-initial-with-ipv6/smpc-consumed.json + ... +``` + +* Alternative 2: create a unique directory for the definition and all the test case stuff: + +``` + /tc1/initial-with-ipv4.yml + /tc1/AAA.xml + /tc1/smpc-consumed.json + /tc2/initial-with-ipv6.yml + /tc2/AAA.xml + /tc2/smpc-consumed.json + ... +``` + +The last one seems to be better and cleaner alternative. + +### Test case definition + +A test case is a yaml list of steps. Each step is a dictionary object with mandatory keys called 'action' and 'arguments'. +An action could be generic (no agent id specified) or agent-related. +Arguments depends on the type of action and are documented at ```'agents/