Using Model Differencing for Architecture-level Regression Testing

HenryMuccini

DipartimentodiInformaticaUniversityofL’AquilaViaVetoio,1-L’Aquila,Italy

muccini@di.univaq.it

Abstract

Regressiontestingcanbesystematicallyappliedatthesoftwarearchitecturelevelinordertoreducethecostofretestingmodifiedsystems,andalsotoassesstheregressiontestabilityoftheevolvedsystem.Withtheadventofmodel-basedspecificationandanal-ysisofsoftwarearchitectures,regressiontestingatthear-chitecturallevelcanbehandledbyanalyzinghowthear-chitecturalmodelevolveswhenmovingfromaninitialtoasubsequentversion.Thispaperanalyzeshowmodeldifferencing,arecentresearchtopicinthemodel-basedcommunity,canbeem-ployedforimplementingmodel-basedarchitecture-levelre-gressiontesting.evolvingversionsofagivenOOsystem,goalofthispaperistomakeuseofmodeldifferencingtechniquesforsoftwarearchitecture-basedregressiontestingofComponent-basedSystems(CBS).Goalofanarchitecture-basedtestingtechniqueistoin-creasetheconfidenceonthequalityofanassemblyofcomponents,byverifyingthatasystemderivedbyassem-blingblack-boxcomponentsconformstoarchitecturalde-cisions[14].Suitesofabstracttestcasesareselectedfromthearchitecturalmodels;testcasesarethenexecutedontheCBSsotoprove(ordisprove)theCBSconformancetoitsarchitecture(hereafterreferredasCBSA)[5,20,7,19,9,21,11,14,15].Figure1summarizesstateoftheartinarchitecture-basedtesting.Goalofanarchitecture-basedregressiontestingtech-niqueistoreiteratethetestingprocessinacosteffectivemanner,wheneveraCBSAchangeovertime[15].Testcasesareselectedbyappropriatelylookingintoexistingtestcases.Newtestcasescanbeadded.Goalofthispaperistoproposeanarchitecture-basedconformanceregressiontestingapproachbasedonmodeldifferences.Theapproachisarchitecture-basedsinceab-stracttestcasesareselected(andre-selected)startingfromanarchitecture-levelspecification,refinedintoconcretetestcases,andrunovertheCBimplementation.Theapproachisbasedonmodeldifferencessincethealgorithmtore-selecttests(tobere-executedonthemodifiedarchitectures)isbasedonmodeldifferencesinformation.WhileaSA-basedregressiontestingtechniqueshallcopewithtwomaintypesofevolution(architecturalevolutionandcodeevolu-tion,see[15])thispaperfocussesonarchitecturalevolution.Therestofthispaperisstructuredasfollows:Section2providessomebackgroundinformationonregressiontest-ing.Section3brieflydescribesandmotivatestheapproach.Section4discussesthedevelopedapproachandappliesitoverarunningexample.Section5brieflydescribesexist-ingtoolsupport.Section7concludesthepaperandoutlinesfuturework.11Introduction

Model-baseddifferencingisbeingrecentlystudiedasawaytodealwithevolvingmodelsandtheirsubsumedsys-tems:giventwomodels(typically,representingtheevolu-tionofthesamesoftwaresystem)modeldifferencingalgo-rithmspermittodiscoversimilaritiesanddifferencesamongthetwomodels.UMLDiff[25],basedontheUMLmeta-model,considersagraphrepresentationofamodel.GivenmodelsAandB,thetwographsaretraversedanddif-ferencesareidentifiedaccordingto“name”similarityand“structure”similarity.TheSiDiffgenericdifferencealgo-rithm[12],instead,calculatesdifferencesoftwomodelsgivenasXMLfiles.UMLdocumentsaretreatedasor-deredtrees(sotoimproveperformance)andwrittenaccord-ingtoaproposedgenericmeta-model.ToolspecificUMLdifferencingalgorithmshavebeenproposedbyUMLtoolvendors,likeIBM/Rational[1].Modeldifferencingtech-niques,basedonmodeltransformationcomposition,havebeenproposedin[6].Whilemodeldifferencingtechniqueshavebeenpro-posedsofarwiththeprimaryintentofreasoningabout33rd EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA 2007)0-7695-2977-1/07 $25.00 © 2007

Bertolino󰀀In󰀀ve󰀀rardi󰀀‘󰀀96󰀀[5]󰀀Richardson󰀀Eickelmann󰀀Richardson󰀀Wolf󰀀‘96󰀀‘󰀀96󰀀[7]󰀀[20]󰀀Richardson󰀀Stafford󰀀Wolf󰀀‘97󰀀[19]󰀀++󰀀+󰀀++󰀀+󰀀Harrold󰀀‘98󰀀[9]󰀀Jin󰀀Rosenblum󰀀Offutt󰀀‘98󰀀‘󰀀01󰀀[21]󰀀[11]󰀀Bertolino󰀀Inverardi󰀀Muccini󰀀et. al.󰀀’󰀀97󰀀-󰀀‘04󰀀[14]󰀀++󰀀+󰀀++󰀀++󰀀++󰀀Muccini,󰀀Dias,󰀀Richards󰀀on󰀀‘05󰀀-󰀀‘06󰀀[15]󰀀Introduction to the󰀀topic󰀀Test case Selection󰀀Testin󰀀gCoverage󰀀Criteria󰀀Activities󰀀Adequacy criteria󰀀Test Execution󰀀Results Evaluation󰀀Unit󰀀T󰀀esting󰀀Integration󰀀Phases󰀀System󰀀Testing󰀀SA󰀀-󰀀based Testing󰀀Goal󰀀SA assessment󰀀Other󰀀Testability󰀀SA styles󰀀Regression Testing󰀀Traceabi󰀀lity󰀀++󰀀++󰀀+󰀀++󰀀+󰀀+󰀀++󰀀+󰀀+󰀀++󰀀+󰀀+󰀀++󰀀++󰀀++󰀀+󰀀+󰀀++󰀀++󰀀+󰀀+󰀀+󰀀+󰀀++󰀀+󰀀+󰀀++󰀀++󰀀+󰀀++󰀀++󰀀+󰀀+󰀀++󰀀++󰀀+󰀀++󰀀Figure1.SoftwareArchitecture-basedTestingTechniques2Background:RegressionTesting

Regressiontesting,asquotedfrom[10],“attemptstoval-idatemodifiedsoftwareandensurethatnonewerrorsareintroducedintopreviouslytestedcode”.Thetraditionalap-proachisdecomposedintotwokeyphases:i)testingtheprogramPwithrespecttoaspecifiedtestsuiteT,andii)whenanewversionP󰀁isreleased,regressiontestingofthemodifiedversionP󰀁toprovideconfidencethatP󰀁iscorrectwithrespecttoatestsetT󰀁.Toexplainhowaregressiontestingtechniqueworksingeneral,letusassumethataprogramPhasbeentestedwithrespecttoatestsetT.WhenanewversionP󰀁isreleased,regressiontestingtechniquesprovideacertainconfidencethatP󰀁iscorrectwithrespecttoatestsetT󰀁.Inthesimplestregressiontestingtechnique,calledretestall,T󰀁containsallthetestcasesinT,andP󰀁isrunonT󰀁.Inselectiveregressiontesting,T󰀁isselectedasa“relevant”subsetofT,wheret∈TisrelevantforP󰀁ifthereisthepotentialthatitcouldproducedifferentresultsonP󰀁thatitdidonP(followingasafedefinition).IngeneraltermsandassumingthatPisaprogramundertest,TatestsuiteforP,P󰀁amodifiedversionofPandT󰀁thenewtestsuiteforP󰀁,regressiontestingtechniquesworkaccordingtothefollowingsteps:1)selectT󰀁,subsetofTandrelevantforP󰀁;2)testP󰀁withrespecttoT󰀁;3)ifneces-sary,createT󰀁󰀁,totestnewfunctionality/structureinP󰀁;4)testP󰀁withrespecttoT󰀁󰀁;5)createT󰀁󰀁󰀁,anewtestsuiteandtesthistory.Allofthesestepsareimportantforthesuccessofaselec-tiveregressiontestingtechniqueandeachoftheminvolvesimportantproblems[8].However,step1(alsocalled,re-gressiontestselection)characterizesaselectiveregressiontestingtechnique.Forthisreason,thispaperfocussesonthisstep.3ApproachOverviewandMotivations

Anycomponent-basedsystemhasitsownarchitecture(CBSA),intermsofarchitecturalelementsandconstraints.TheCBSAmodelspecifies,byfocussingoncomponentsintegrationwhilehidingimplementationdetails,whattoex-pect(intermsofpropertiesandqualities)fromtheassem-blyofcomponents.Sinceanarchitecturalmodelmaycon-sistofdifferentdiagrams,accordingtothemulti-viewar-chitecturalmodelingphilosophy[13],wedoherefocusonstructuralandbehavioraldiagrams,thatis,thosetypicallyutilizedformodelingtheSAtopology(i.e.,componentdia-grams)anditsbehaviorintermsofinteractingcomponentsandconnectors(i.e.,sequenceandstatediagrams).FromtheCBSA,architecture-leveltestcasescanbeex-tracted,sotobesuccessivelyrunontheCBSimplementa-tionforcomplianceanalysis.Whenthesoftwarearchitec-tureevolves,themodeldocumentingthearchitecturaldeci-sionsevolvesitself.Architecturalelementscanbeadded,removed,ormodified.Thearchitectureitselfcanberecon-figured,bychangingconnectivityamongarchitecturalele-ments.ByfollowingthedrawinginFigure2,anewversionoftheCBSAmodelisthencreated.Theapproachweareproposingmakesuseofamodeldifferencingalgorithmandtoolwedeveloped,inordertoidentifyhowthestructuralandbehavioralmodelstakenintoconsiderationvary.Theoutputofthismodeldifferencinganalysisisadeltadiagramwhichistakenininputbyatestre-selectionalgorithminordertoidentifywhichtestcasesneedtobere-tested.Thisapproachinheritsfromexistingmodel-differencingalgorithmsthemainprinciplesof“name”and“structure”33rd EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA 2007)0-7695-2977-1/07 $25.00 © 2007

CBSA model, v1󰀀

evolves󰀀CBSA model, v2󰀀

4.1TheATMrunningexampleWemakeuseoftheATMsystemastherunningex-ampleforthispaper.Whilemanyarchitecturedescriptionlanguageshavebeenproposedsofarforsoftwarearchi-tecturemodeling,inthispaperwecastourstudyontheCHARMY[4,18]notationformodelingsoftwarearchitec-tures.CHARMYallowsthespecificationofasoftwarearchitec-turebymeansofbothatopological(static)descriptionandabehavioral(dynamic)one.Todescribethearchitecturaltopology,CHARMYusesasubsetoftheUMLcomponentdiagram.Anarchitecturalcomponentisdrawnwiththefa-miliarUML2.0notationforcomponents.Aconnectorcanbeseenasacomplexcoordinationelementorasasimplecommunicationchannel.ComplexconnectorsaremodeledusingtheUMLnotationforcomponents,whileanarchitec-turalchannelisrepresentedbyanassociationlinebetweenarchitecturalcomponents.ThearchitectureoftheATMsystem(calledATMv2)thatweconsideriscomposedoffourcomponentsasshowninFigure3:theUser,thetransactionmanager(TMcompo-nent),thebankaccount(BAcomponent),andtheauthenti-cation(AUTHcomponent).TheUsercomponentcommuni-catesonlywiththeTMcomponentthatforwardstheservicerequeststotheBAcomponentortotheAuthcomponent.TheinternalbehaviorofeachcomponentisspecifiedintermsofaStateTransitionDiagram(STD)notationclosetothePromelasyntax,definedasaquadruple(S,L,S0,T),whereSisthesetofstates,Listhesetofdistinguishedlabels(actions)denotingtheSTDalphabet,S0∈Sisthelinitialstate,andT={−→⊆S×S|l∈L}isthetransi-tionrelationlabeledwithelementsofL.ThelabelsLarestructuredasfollows:[guard]‘/‘event‘(‘parameterlist‘)“/‘op1‘;‘···‘;‘opnwhereguardisabooleanconditionthatdenotesthetransi-tionactivation,aneventecanbeamessagesentorreceived(denotedbyanexclamationmark“!e”oraquestionmark“?e”,respectively)oraninternaloperation(“e”),andcanhaveseveralparametersasdefinedintheparameterslist.op1,op2,···,opnaretheoperationstobeexecutedwhenthetransitionfires.ThenotationisshowninFigure4:theUsercomponenthandlesthreedifferentrequests,onefortheauthentication(!login)followedbytwopossibleresponses(?loginokand?loginko),oneforwithdrawingmoneyfromheraccount(!withdraw),andoneforrechargingthemobilephonecredit(!chargePhone).TheTMcomponentcontainsthelogicoftheATMsystem.Thiscomponentreceivesthelo-ginrequestfromtheUser(?login)andforwardsittotheAUTHcomponent(!loginAuth).Twoarethepossiblere-sponsesthatTMcanreceivefromAUTH:loginsuccessModel󰀀Differencing󰀀Algorithm󰀀Test Case for󰀀CBSA, v1󰀀Test󰀀re-selection󰀀Algorithm󰀀Test Case for󰀀CBSA, v2󰀀Figure2.ApproachOverviewsimilarityheuristics,“XML-baseddifference”calculation,modeldifference“coloring”.Itdiffersfrompreviousworkfortwomainreasons:wedoimplementthedifferencealgorithmoverstatemachines(whilepreviousworkfocusonclassdiagrams),anditisthefirstregressiontestingapproachexplicitlysupportedbymodeldifferencingalgorithms.Atthebestofourknowl-edge,whileinitialreasoningonSA-basedregressiontest-ingtechniqueshavebeenproposedin[9],SARTE(Soft-wareArchitecture-basedRegressionTesting)istheonlyframeworkexplicitlydealingwithSA-basedregressiontest-ing[15].ThispaperinheritsfromSARTEthemaincon-ceptsandextendsthetestre-selectionphasethroughmodeldifferencing.Inthedomainofmodel-basedregressiontest-ing,onlyveryfewapproachesandtoolssupportregressiontesting[16]andthosetoolsworkatthelowleveldesign,notatthearchitecturallevel.Indeed,otherworkonsoftwarearchitecture-basedanalysisanddependenceanalysisatthearchitecturallevel[23]mayberelatedtothisapproach.4UsingModelDifferencingforArchitecture-levelRegressionTesting

Intherestofthispaper,whileassumingtheCBSAmodelisavailableandarchitecturaltestcaseshavebeenalreadyselected,wewillfocusonthemodeldifferencingalgorithm(usedtodetectdifferencesandsimilarities),andtothethetestre-selectionalgorithm(usedtoidentifywhicharchitec-turaltestcasesarestillvalidinthemodifiedversionoftheCBSA).Therestofthepaperwillfocusonthetwoalgorithms.WeinitiallydescribetheATMsystemtobeusedasarun-ningexample,thenproposethosetwoalgorithms.33rd EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA 2007)0-7695-2977-1/07 $25.00 © 2007

(?loginauthok),andloginfailure(?loginauthko).Incaseofsuccess,theUserishabilitatedtoavailableservices(i.e.,withdrawmoneyorrechargemobilephone).TMre-ceivestheresponseforbothservicesandforwardsthemtotheUsercomponent.Theothertwocomponents,BAandAUTH,managethebankaccountservices(i.e.,withdraw,charge)andloginservices,respectively.Asremarkedbefore,thoseareabstracttestspecifica-tions(asinmostofthemodel-basedtestingapproaches)tobesuccessively(i.e.,whentheCBSimplementationbe-comesavailable)refinedintoconcreteandexecutabletestcases.Theycanbeselectedandspecifiedaccordingtoexistingmodel-basedandarchitecture-basedtestingtech-niques[16,14].4.2ModelDifferencingWhilethecomparisonbetweentopologydiagramsisastraightforwardinstantiationandadaptationofexistingal-gorithms,tokeepthesectionmorefocussed,wehereana-lyzehowwecomparedtheCHARMYstatemachines.Whendealingwithstatemachines,twoarethemainel-ementstobecompared:statesandtransitions.AsshownintheexcerptoftheCHARMYXMLschemaforstatedi-agramsinFigure5,astateSintheSTDgraph(ELE-MENTSTATEtag)isidentifiedbyanID,aNAME,andaTYPE(initialorinternalstate).AtransitionlabelLhasamoreinformativestructure,withanID,aNAME,atransitionTYPE(synchronous,asynchronous,orloop),thetransitionSOURCE,thetransitionTARGET,theACTIONS(events),theCONDITIONS(guards),andPARAMETERS.Figure3.TopologyDiagramoftheATMexam-pleFromtheATMarchitecturalspecification,asuiteofseventestspecificationshavebeenselected,byapplyingtheapproachdescribedin[14].TheselectedcoveragecriteriarequiresthecoverageofeachedgeintheSTDgraph.Infor-mally,thetestspecificationsare:t1:theUsertriestologin,butheraccessisdenied;t2:theUserlogins,thenlogouts;t3:theUserlogins,askstowithdrawmoney,theconnec-tionwithherbankaccountfailsandshecannotwith-drawmoney;t4:theUserlogins,askstochargeherphone,thecon-nectiontothebankaccountsucceed,butshehasnotenoughmoneyinhercheckingaccount,andtheoper-ationfails;t5:theUserlogins,askstochargeherphone,andsucceed.t6:theUserlogins,askstowithdrawmoney,theconnec-tionwithherbanksucceed,buthehasnotenoughmoneyforthewithdraw;t7:theUserlogins,askstowithdrawmoney,andsuc-ceeds.󰀀󰀀󰀀󰀀󰀀ID󰀀NAME󰀀TYPE󰀀󰀀󰀀ID󰀀NAME󰀀TYPE󰀀SOURCE󰀀TARGET󰀀ACTIONS󰀀CONDITIONS󰀀PARAMETERS󰀀Figure5.TheCHARMYstatemachinesXMLschemaWhencomparingtwoCHARMYstatediagrams,theal-gorithmtakesininputthetwoSTDgraphsandoutputsanSTDdiffdifferencinggraphhighlightingunchangedandchangedelements.ThealgorithminitiallycomparesELE-MENTSTATEtagsforstatenamessimilarity.STD1ELE-MENTSTATEtagsaretraversedandcomparedwithSTD2states.Twostates,SiinSTD1andSjinSTD2,arethesame33rd EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA 2007)0-7695-2977-1/07 $25.00 © 2007

Figure4.StatediagramsoftheATMcomponentsifthreeconditionshold:i)SiandSjhavethesamenames,ii)theyhavethesameincomingandoutgoingtransitions(sametransitionnames),andiii)theyhavethesamesourceandtargetstates.Wheneverthesethreeconditionsapply,thestateiscopiedinSTDdiffandcoloredingreen.Ifatleastoneconditionapplies(butnotallofthem),thenthestateismarkedaschangedandcoloredinyellow.IfstateSiinSTD1hasnotanequivalentstateinSTD2,thenSihasbeenremovedwhentransitingfromSTD1toSTD2.If,in-stead,SiinSTD2hasnotanequivalentstateinSTD1,thenSihasbeenaddedwhentransitingfromSTD1toSTD2.Addedstatesarecoloredinred,whileremovedonesarerepresentedasdottedcircles.Procedure:stateNameComparisoninput:STD_i,STD_i+1output:STD_diff

Repeat

s_i=getElementState(STD_i);

markAsVisited(s_i,STD_i);ifexists(s_i,STD_i+1)

thenmarkAsVisited(s_i,STD_i+1);if(getName(s_i,STD_i)!=

getName(s_i,STD_i+1))

thenaddAsChanged(s_i,STD_diff)elseif(getSource(s_i,STD_i)=

getSource(s_i,STD_i+1)&(getTarget(s_i,STD_i)=

getTarget(s_i,STD_i+1))

thenaddAsSame(s_i,STD_diff)

elseaddAsRemoved(s_i,STD_diff)UntilnotVisited(STD_i)=!emptyifnotVisited(STD_i+1)=!empty

thenforeach(s_i,notVisited(STD_i+1))

addAsNew(s_i,STD_diff)

Afterstatenamecomparison,thealgorithmproceedswithtransitionnamesimilarity.STD1ELEMENTMES-SAGEtagsaretraversedandcomparedwithSTD2transi-tions.Inordertoidentifytransitionequivalence,twotran-33rd EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA 2007)0-7695-2977-1/07 $25.00 © 2007

sitionsTiandTjarethesameiftwoconditionshold:i)thetransitionlabelisthesame(ACTIONS,CONDITIONS,andPARAMETERStags),andii)thetargetandsourcestatesarethesame(SOURCEandTARGETtags).Anonmodifiedtransitionisrepresentedingreen.Incaseonlyonecondi-tionholds,thetransitionhasbeenmodifiedanditiscoloredinyellow.Addedtransitionsarecoloredinred,whilere-movedonesarerepresentedviadottedlines.Aversion3oftheATMexamplehasbeenproduced(ATMv3),wheretheUsercanalsocheckhermoneybal-ance,andtheTMhasadifferentconnectionmanagement:first,aconnectionwiththeBAisestablished,thenthewith-draw,charge,ormoneybalanceservicesareactivatedac-cordingtotheUserrequest(whileinthepreviousversion,whenaserviceisactivated,itopensitsownconnection).ByapplyingthemodeldifferencingalgorithmtotheTMdiffcomponent,weobtainedthediagraminFigure6.Ingreen,statesandtransitionspreserved.Inred,newstatesandtran-sitions.Inyellow,modifiedstatesandtransitions.BytakingacloserlooktotheTMdiffdiagram,wecanmakethefollowingobservations:changeokandchangekostatesareingreensincetheyhavesamenames,samein-comingandoutgoingtransitions,andsamesourceandtar-getstatesinthetwoversions.Connectingisamodifiedstate(i.e.,inyellow)since,evenifsourceandtargetstatesareunchanged,theydohavemodifiedincomingtransitionsandnames.Balanceisanewstate(thusinred)sincethereisnostatewiththesamename,andincomingandoutgoingtransitions,andsourceandtargetstatediffer.?Connectokisanonmodifiedtransition(i.e.,ingreen)sincebothnamesandsourceandtargetstatesarethesame.?ChangePhoneismodifiedsincethesourcestatediffersinthetwoversions.!Moneyreportisanewtransition(thusinred)sincethereisnotransitionwithsuchnameinthefirstversion,andthereisnoothertransitionwiththesamesourceandtargetstates.4.3TestCasere-SelectionInselectiveregressiontesting,givenPasoftwaresystem,Pitsevolution,andTatestsuiteforP,t∈Tisselectedtobere-executedinP󰀁(i.e.,t∈T󰀁)ifthereisthepotentialthatitcouldproducedifferentresultsonP󰀁thatitdidonP(fol-lowingasafedefinition).Dangerousregionsareidentifiedtodenoteportionsofthesystemthat,iftraversed,mayleadtoadifferentbehaviorinPandP󰀁[17].Wheneveratestcaset∈TcausesPtotraverseadangerousregion,ttra-versesamodifiedcodethatmaycauseadifferentbehaviorinPandP󰀁,andtmustbere-runonT󰀁.Ourapproachidentifiesdangerousregionsbymodeldif-ferencing.Anynode/arcintheSTDdiffgraphwhichhasbeencoloredinred,yellow,orviadottedlinesrepresentdangerousregions.Thetestcasere-selectionalgorithmpresentedinthis󰀁sectionperformstwodifferentactions,thusimplementingsteps1and3inSection2:i)itcheckswhetheranexist-ingtestcaset∈Tmustbere-testedinT󰀁(regressiontestselectionphase),andii)itidentifiesnewtestcasest󰀁󰀁nec-essarytocoveraddedportionsoftheCBSA(newtestcaseselectionphase).Theregressiontestselectionphaseisperformedbysim-ulatingeachtestcaset∈TintheSTDdiffgraph.Wheneverthetestcasecoversyellowordottedstatesandtransitions,thenitneedstobere-runonP󰀁.Wheneveritcoversredstatesortransitions,newtestcasesmightbeneededtotesttheaddedportionoftheCBSA.BytakingintoconsiderationtheATMexample,andtheseventestcasespreviouslydescribed,wenoticethatmostofthemneedtobere-run.BysolelyfocusontheTMcom-ponent,testcasest1andt2donotneedtobere-testedsincetheycoveronlygreenstatesandtransitions(theupperleftpartoftheTMdiffgraphinFigure6).t3needstobere-tested,sinceittraversesthe!connecttransitionmarkedinredandtheconnectingstateinyellow.Sinceanyothertestcaset4tot7traversesthe!connecttransitionandthecon-nectingstate,theyneedtobere-testedall.ThisisduetothefactthattheTMcomponentbehaviorhasbeenconceptuallyrestructured(connectionfirst,thananyoftheservice).As-sumingtheonlychangefromversionATMv2toATMv3wouldhavebeenaddingthecheckmoneybalancefeature,noneoftheseventestcaseswouldhavebeenre-tested.Thenewtestcaseselectionphaseisinsteadimplementedbygeneratingtestcasescoveringtheaddedstatesandtran-sitions.IntheATMv3specification,thenewstatesarebalance,waiting,report,andcheck1,whilethenewtransi-tionsare!connect,?moneybalance,!moneyreport,?report,andprint.Byre-applyingthesamecoveragecriteriaap-pliedintheinitialgraph(i.e.,anyedgeintheSTDgraph),!connectisalreadycoveredbythetestcasest3-t7,newtestcasesareneededfortheothertransitions.Twotestcasesareneededforcoveringallthenewtransitions:t8󰀁–theUserlogins,getconnected,asksformoneybalance,andobtainsaprintedstatement,andt9󰀁–theUserlogins,getconnected,asksformoneybalance,butdoesnotreceiveaprintedstate-mentsincethereisnopaperintheATMmachine.Insum-mary,seventestcasesneedtoberunonATMv3:t3-t7,t8󰀁,andt9󰀁.5Toolsupport

Theregressiontestingmethodhasbeenimplementedin-sidetheCHARMYframeworkthroughthreemainmodules.TheCHARMYstandardeditorpluginwhichallowsthemodel-basedspecificationoftheCBSAunderanalysis.ItallowsthespecificationofaCBSAthroughatopologyed-itorandabehavioraldiagram(assummarizedinSection4andshowninFigures3and4).33rd EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA 2007)0-7695-2977-1/07 $25.00 © 2007

Figure6.ModifiedStatediagramfortheTMcomponentTheCHARMYarchitecturaltestingplugin,recentlyde-veloped,whichallowsthetestcasesselectionfromthear-chitecturalspecification.Thismoduleimplementstheap-proachin[14]bygeneratingaglobalautomatonoutofthecomponentstatediagrams,byapplyinganobservationsotofocusonrelevant(withrespecttoatestpurpose)por-tionsofthegraph,andbyimplementingvariouscover-agecriteria[24,19](alledges,allcomponents,allstates,McCabe).TheCADP(ConstructionandAnalysisofDis-tributedProcesses)[2]toolfeatureshavebeenutilizedinmanystepsinthetestcaseselectionprocess,andintegratedwithCHARMY.TheCHARMYregressiontestingplugin,currentlyinitsbetaversion,whichimplementsthemodeldifferencingandtestre-selectionalgorithmsdescribedinthispaper.Thepro-ducedoutputisacoloredgraphhighlightingmodeldiffer-ences(astheoneinFigure6)andasetofre-selectedtestcases.ingtechniques.6InitialConsiderations

Thisinitialworkonmodeldifferencing-basedregressiontestingseemstobeverypromising,duetothecurrentinter-estonmodel-basedspecificationsofsoftwarearchitectures(seee.g.,[3])andtotheongoingworkonmodeldifferenc-However,manyissuesstillneedtoberesolved.Firstofall,itisimportanttounderstandhowmuchthealgorithmsareusefulandeffectiveforselectiveregressiontesting.Ac-cordingto[22]fivearethemaincategorieswhichshouldbetakenintoaccountwhenevaluatingselectivereteststrate-gies:inclusiveness,precision,efficiency,generality,andac-countability.Morestudyisnecessaryinordertobeabletopositionourapproachaccordingtothoseparameters.However,initialstudyletusbelievethatasfarasconcernprecision(whichmeasurestheabilityofaRTtechniquetoomitteststhatarenon-modification-revealing)thepro-posedtechniqueissafe(i.e.,itre-selectsanyrelevanttestcase)evenifitsinclusiveness(whichmeasurestheextenttowhichaselectivereteststrategychoosesmodificationre-vealingtests)canbeimprovedbymakingthealgorithmsconsideringfinergrainedarchitecturalchanges.Thisap-proachconsidersonlytheimpactofarchitecturalchangesonthetestcasere-selectionprocess,whilenottakingintoaccountcodechanges.Moveover,thebasicassumptionofourapproachisthatitispossibletotracearchitecturalcom-ponentstorealcomponentsbyfocussingontheconcretearchitectureofaCBS(calledCBSA).33rd EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA 2007)0-7695-2977-1/07 $25.00 © 2007

7ConclusionsandfutureWork

Regressiontestingcanbesystematicallyappliedatthesoftwarearchitecturelevelinordertoanticipatethetestingphaseassoonaspossibleinthesoftwaredesignstage,whilereducingthecostofretestingmodifiedsystems.Whilepre-viousefforthasinvestigatedwhatregressiontestingatthearchitecturallevelmeans[9]andhowitcanbeconceptu-allyimplementedinatestingandregressiontestingframe-work[15],thispaperhasshownhowtheregressiontestse-lectionandthenewtestcaseselectionphasescanbeimple-mentedthroughmodeldifferencing.Thispaperhasoutlinedtwoalgorithms:oneformodeldifferencingofCHARMYstatemachines,onefortestre-selectionbasedonthemodeldifferencingresults.InitialresultshavebeenshowninthecontextoftheATMrunningexample.Infutureworkwedoplantoimprovetoolsupportandtoapplyourapproachtoacomplexexample.Fromaconcep-tualviewpoint,wewanttounderstandhowthisapproachcanbeappliedwhensubsequentevolutionsoftheCBSAapply(linearderivation),orwhenthesameversionevolvesaccordingtodifferentevolutionbranches(branchingderiva-tion).Acknowledgment

TheauthorofthispaperwishtothankLindaCorsettiwhoimplementedpartofthemodeldifferencingregressiontestingapproachandtheanonymousreviewerswhopro-videdrelevantcommentsandsuggestionsonhowtoim-proveit.References

[1]ComparingandmergingUMLmodelsinIBM

RationalSoftwareArchitect.http://www-128.ibm.com/developerworks/rational/library/05/712comp/.[2]ConstructionandAnalysisofDistributedProcesses(CADP)

projecthomepage.http://www.inrialpes.fr/vasy/cadp/.

[3]TheSAEArchitectureAnalysisandDesignLanguage

(AADL).http://www.aadl.info/.[4]CHARMYProject.CharmyWebSite.

http://www.di.univaq.it/charmy,2004.

[5]A.BertolinoandP.Inverardi.Architecture-basedSoftware

Testing.InProc.ISAW96,October1996.

[6]A.Cicchetti,D.D.Ruscio,andA.Pierantonio.ADSMLfor

ModelDifferences.InEuropeanWorkshoponCompositionofModelTransformations,CMT2006,2006.

[7]N.EickelmanandD.Richardson.WhatMakesOneSoft-wareArchitectureMoreTestableThanAnother?InInProc.ISAW-2,October1996.

[8]T.L.Graves,M.J.Harrold,J.-M.Kim,A.Porter,and

G.Rothermel.AnEmpiricalStudyofRegressionTestSelec-tionTechniques.InProc.ofthe20thInt.Conf.onSoftwareEngineering(ICSE’98),pages188–197,Japan,April1998.

[9]M.J.Harrold.Architecture-BasedRegressionTestingof

EvolvingSystems.InProc.Int.WorkshopontheRoleofSoftwareArchitectureinTestingandAnalysis-ROSATEA,pages73–77,July1998.

[10]M.J.Harrold.Testing:ARoadmap.InA.Finkelstein,ed-itor,ACMICSE2000,TheFutureofSoftwareEngineering,pages61–72,2000.

[11]Z.JinandJ.Offutt.Derivingtestsfromsoftwarearchitec-tures.InISSRE,pages308–313,2001.

[12]U.Kelter,J.Wehren,andJ.Niere.AGenericDifference

AlgorithmforUMLModels.InProceedingsoftheSE2005,Essen,Germany,March2005.

[13]P.Kruchten.ArchitecturalBlueprints-The“4+1”View

ModelofSoftwareArchitecture.IEEESoftware,12(6):42–50,November1995.

[14]H.Muccini,A.Bertolino,andP.Inverardi.UsingSoftware

ArchitectureforCodeTesting.IEEETrans.onSoftwareEn-gineering,30(3):160–171,March2003.

[15]H.Muccini,M.Dias,andD.J.Richardson.Software

Architecture-basedRegressionTesting.Int.JournalofSys-temsandSoftware,specialissueonArchitectingDepend-ableSystems,Toappear2006.

[16]L.Naslavsky,D.Richardson,andH.Ziv.Scenario-based

andStateMachine-basedTesting:AnEvaluationofAuto-matedApproaches.Technicalreport,UniversityofCalifor-nia,Irvine-ISRTRUCI-ISR-06-13,2006.

[17]A.Orso,N.Shi,andM.J.Harrold.Scalingregressiontest-ingtolargesoftwaresystems.InProceedingsofthe12thACMSIGSOFTSymposiumontheFoundationsofSoftwareEngineering(FSE2004),pages241–252,2004.

[18]P.Pelliccione.CHARMY:AframeworkforSoftwareArchi-tectureSpecificationandAnalysis.PhDthesis,ComputerScienceDept.,UniversityofL’Aquila,May2005.

[19]D.J.Richardson,J.Stafford,andA.L.Wolf.AFormalAp-proachtoArchitecture-basedSoftwareTesting.Technicalreport,UniversityofCalifornia,Irvine,1998.

[20]D.J.RichardsonandA.L.Wolf.SoftwareTestingatthe

ArchitecturalLevel.InISAW-2,inJointProc.oftheACMSIGSOFT’96Workshops,pages68–71,1996.

[21]D.Rosenblum.ChallengesinExploitingArchitecturalMod-elsforSoftwareTesting.InProc.Int.WorkshopontheRoleofSoftwareArchitectureinTestingandAnalysis-ROSATEA,July1998.

[22]G.RothermelandM.J.Harrold.AFrameworkforEvaluat-ingRegressionTestSelectionTechniques.InInProc.16thInt.ConferenceonSoftwareEngineering,ICSE1994,pages201–210,Sorrento,Italy,May1994.

[23]J.A.StaffordandA.L.Wolf.Architecture-leveldependence

analysisinsupportofsoftwaremaintenance.InISAW’98:ProceedingsofthethirdinternationalworkshoponSoftwarearchitecture,pages129–132,NewYork,NY,USA,1998.[24]A.H.WatsonandT.J.McCabe.StructuredTesting:ATest-ingMethodologyUsingtheCyclomaticComplexityMetric.NISTSpecialPublication500-235,August1996.

[25]Z.XingandE.Stroulia.DifferencinglogicalUMLmodels.

SpecialissueofAutomatedSoftwareEngineeringJournal.SelectedpapersfromASE2005.Toappear.

33rd EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA 2007)0-7695-2977-1/07 $25.00 © 2007