Loading...

nunit-discuss@googlegroups.com

[Prev] Thread [Next]  |  [Prev] Date [Next]

[nunit-discuss] Re: Only can run entire suite of tests on 2.6 nickdu Thu Apr 05 13:01:19 2012

The first thing I did was to create a console application which
references the assembly that contains the tests and it calls the Lock
Test3 and Test5.  These are minimum set of tests I would run from
NUnit GUI from choosing the Lock class that would fail.  From my
console app they both ran fine.  Here is my console application:

using System;
using Redbone.Framework.BusinessProcess.Tests;

public class Application
{
        public static void Main()
        {
                try
                        {
                        Lock @lock = new Lock();
                        @lock.Test3();
                        @lock.Test5();
                        Console.WriteLine("Completed both tests successfully.");
                        }
                catch(Exception e)
                        {
                        Console.WriteLine(e.ToString());
                        }
        }
}

Now I'll try to come up with a minimum set of code which fails in
NUnit GUI which I could send out to others.

By the way, here is the details of the exception the debugger gave me
when the exception was thrown in the nunit-agent:

System.TypeInitializationException occurred
  Message=The type initializer for '<Module>' threw an exception.
  Source=Redbone.Framework.BusinessProcess
  TypeName=<Module>
  StackTrace:
       at
Redbone.Framework.BusinessProcess.ProcessService.Start(XmlQualifiedName
name, String description, Version version, String step, String key,
IRow row)
       at Redbone.Framework.BusinessProcess.Tests.Lock.Test5() in C:
\data\development\Redbone\bpf\Tests\Lock.cs:line 76
  InnerException: System.TypeInitializationException
       Message=The type initializer for '<Module>' threw an exception.
       Source=ssoDotNet
       TypeName=<Module>
       StackTrace:
            at
<CrtImplementationDetails>.ThrowModuleLoadException(String ,
Exception )
            at
<CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )
            at .cctor()
       InnerException: <CrtImplementationDetails>.ModuleLoadException
            Message=The C++ module failed to load while attempting to
initialize the default appdomain.

            Source=msvcm90
            StackTrace:
                 at
<CrtImplementationDetails>.ThrowModuleLoadException(String
errorMessage, Exception innerException)
                 at
<CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )
                 at .cctor()
            InnerException:
System.Runtime.Serialization.SerializationException
                 Message=Unable to find assembly
'Redbone.Framework.BusinessProcess.Common, Version=4.0.0.0,
Culture=neutral, PublicKeyToken=e3fe5053ddfa671d'.
                 Source=mscorlib
                 StackTrace:
                   Server stack trace:
                      at
System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssembly()
                      at
System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryAssemblyInfo
assemblyInfo, String name)
                      at
System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor(String
objectName, String[] memberNames, BinaryTypeEnum[] binaryTypeEnumA,
Object[] typeInformationA, Int32[] memberAssemIds, ObjectReader
objectReader, Int32 objectId, BinaryAssemblyInfo assemblyInfo,
SizedArray assemIdToAssemblyTable)
                      at
System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped(BinaryObjectWithMapTyped
record)
                      at
System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
                      at
System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler
handler, __BinaryParser serParser, Boolean fCheck, Boolean
isCrossAppDomain, IMethodCallMessage methodCallMessage)
                      at
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream
serializationStream, HeaderHandler handler, Boolean fCheck, Boolean
isCrossAppDomain, IMethodCallMessage methodCallMessage)
                      at
System.Runtime.Remoting.Channels.CrossAppDomainSerializer.DeserializeObject(MemoryStream
stm)
                      at
System.Runtime.Remoting.Messaging.SmuggledMethodCallMessage.FixupForNewAppDomain()
                      at
System.Runtime.Remoting.Channels.CrossAppDomainSink.DoDispatch(Byte[]
reqStmBuff, SmuggledMethodCallMessage smuggledMcm,
SmuggledMethodReturnMessage& smuggledMrm)
                      at
System.Runtime.Remoting.Channels.CrossAppDomainSink.DoTransitionDispatchCallback(Object[]
args)
                   Exception rethrown at [0]:
                      at
System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage
reqMsg, IMessage retMsg)
                      at
System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData&
msgData, Int32 type)
                      at System.AppDomain.get_Id()
                      at
<CrtImplementationDetails>.DoCallBackInDefaultDomain(IntPtr function,
Void* cookie)
                      at
<CrtImplementationDetails>.DefaultDomain.Initialize()
                      at
<CrtImplementationDetails>.LanguageSupport.InitializeDefaultAppDomain(LanguageSupport*
 )
                      at
<CrtImplementationDetails>.LanguageSupport._Initialize(LanguageSupport* )
                      at
<CrtImplementationDetails>.LanguageSupport.Initialize(LanguageSupport* )
                 InnerException:

Our framework makes use of SSO, some managed TIBCO assemblies.  The
SSO assemblies in turn have implicit links to unmanaged assemblies.
These assemblies are in a directory which are in the path.  I had a
breakpoint on ProcessService.Start() but it never hit that.  It shows
Start() at the top of the exception stack.  My guess is that when the
call was attempted it started to jit the code for Start() and that's
where it ran into a problem.

Thanks,
Nick

On Apr 4, 11:47 am, nickdu <[EMAIL PROTECTED]> wrote:
> I'm going to put some effort in today to get a simple repro of this.
> I hope I'm successful.  By the way, I tried 2.5.10 and the same issue
> occurs so it's not specific to 2.6.
>
> Thanks,
> Nick
>
> On Apr 2, 5:09 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
>
>
>
> > That would help. I'd definitely like to see it.
>
> > Regarding dependencies, I'd say that if running tests 1-4 has an
> > effect on test 5, that's a dependency by definition. It's just one that
> > we don't yet understand and which is probably due to the underlying
> > platform - or NUnit itself - rather than something you coded explicitly.
>
> > Charlie
>
> > On Mon, Apr 2, 2012 at 12:43 PM, nickdu <[EMAIL PROTECTED]> wrote:
> > > Yeah, I'm thinking about spending a little bit of time to create a
> > > skelaton of our framework that has the same dependent assemblies to
> > > see if I can repro and if so send it along or post the code.
>
> > > With respect to test dependencies, I don't think this could be the
> > > case.  As I said, I can start NUnit GUI fresh and run Lock Test5
> > > without a problem, and therefore it's not dependent on running any
> > > other tests.  However, if I start NUnit GUI fresh and select the Lock
> > > 'set' of tests and run them then Test1 - Test4 succeed and Test5 -
> > > Test8 succeed.
>
> > > Nick
>
> > > On Apr 2, 12:54 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
> > >> Hi Nick,
>
> > >> On Mon, Apr 2, 2012 at 6:04 AM, nickdu <[EMAIL PROTECTED]> wrote:
> > >> > There were trace files created with trace information in them, just
> > >> > none of it seemed to point to a reason for the issue I'm having.
>
> > >> Right - that's what I meant. NUnit is simply considering your
> > >> problem as a test failure, not an NUnit problem worthy of logging.
>
> > >> > I don't have any automatic reload setting turned on.
>
> > >> OK
>
> > >> > When I mentioned loading it was more about how NUnit loads a class and/
> > >> > or method in order to execute it.  It seems odd that if I select the
> > >> > Lock class and run I get different behavior than when I choose each
> > >> > Lock test method individually and run them.  My guess is that NUnit is
> > >> > loading the code differently in those two cases.
>
> > >> NUnit's "Loading" involves only your test assembly. It has to happen 
> > >> before
> > >> the tests can be displayed in the Gui. That loading is only repeated if 
> > >> you
> > >> use a reload item on the menu or if you have enabled automatic reload on
> > >> run or when the assembly changes.
>
> > >> Loading of dependent assemblies happens according to normal .NET rules,
> > >> as the code is jitted and executed. Again, once a dependent assembly is
> > >> loaded into the test AppDomain, it stays there.
>
> > >> So, your problem arises when the dependent assembly is first needed by
> > >> one of the Lock tests. If you have run some other test first, which 
> > >> successfully
> > >> loads the assembly, then the Lock tests have no problem. This is exactly
> > >> the sort of inter-test dependency I was originally writing about - not
> > >> necessarily
> > >> a dependency that you intentionally created, but one that exists 
> > >> nevertheless.
>
> > >> I guess that gives you two possible attacks on the problem:
> > >> 1. Trace how and why the Lock class cannot load the assembly.
> > >> 2. Trace how and why the complete class is able to load the assembly.
>
> > >> What's different in these classes?
>
> > >> Charlie
>
> > >> PS: Feel free to post some code if it doesn't expose any company secrets.
>
> > >> > Thanks,
> > >> > Nick
>
> > >> > On Apr 1, 6:08 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
> > >> >> Well, the absence of trace info from NUnit at least tells us that 
> > >> >> NUnit is
> > >> >> not aware of any problem. I guess we knew that though.
>
> > >> >> Regarding loading: NUnit does not reload any tests unless you tell it 
> > >> >> to.
> > >> >> You should probably turn off any automatic reload settings if you have
> > >> >> them set. That way you can manually control when a reload takes place.
>
> > >> >> Charlie
>
> > >> >> On Sun, Apr 1, 2012 at 2:55 PM, nickdu <[EMAIL PROTECTED]> wrote:
> > >> >> > I haven't use NUnit's tracing yet.  I just now tried turning on the
> > >> >> > most verbose level of tracing.  I looked through all the trace files
> > >> >> > generated and there didn't seem to be any information in there which
> > >> >> > pointed to the problem.
>
> > >> >> > Still doing more testing.  As I mentioned before, if I select the 
> > >> >> > Lock
> > >> >> > set of tests, and Lock is a class with a bunch of test methods, and
> > >> >> > run that set, the first four tests, test1 - test4, succeed and the
> > >> >> > last four, test5 - test8, fail.  The first four are negative test
> > >> >> > cases which don't make it past the first assembly implementing our 
> > >> >> > API
> > >> >> > as it validates the parameters and fails.  The last four have valid
> > >> >> > parameters and thus make it farther into our API and require this
> > >> >> > Common assembly be loaded.  This is where it fails.
>
> > >> >> > Interestingly, I can execute test1 successfully and then select 
> > >> >> > test5
> > >> >> > and execute that successfully in the same instance of the NUnit GUI.
> > >> >> > However if I select all Lock test cases it fails.  Does NUnit load
> > >> >> > things differently if you select individual tests version a set of
> > >> >> > tests by choosing the class?
>
> > >> >> > Thanks,
> > >> >> > Nick
>
> > >> >> > On Apr 1, 1:24 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
> > >> >> >> Still, this doesn't explain why your tests have anything at all to 
> > >> >> >> do
> > >> >> >> with the primary AppDomain. NUnit does run all tests in a separate
> > >> >> >> AppDomain and creates a new one each time you reload the tests.
>
> > >> >> >> Have you turned on NUnit's internal trace and looked at whether it
> > >> >> >> tells you anything?
>
> > >> >> >> Charlie
>
> > >> >> >> On Sun, Apr 1, 2012 at 5:50 AM, nickdu <[EMAIL PROTECTED]> wrote:
> > >> >> >> > I did some more tests, and while I found out some new behavior I 
> > >> >> >> > still
> > >> >> >> > don't know what's going on here.
>
> > >> >> >> > In order to test your custom exception suggestion I figured I 
> > >> >> >> > would
> > >> >> >> > take one of the tests that failed and catch all exceptions and 
> > >> >> >> > turn
> > >> >> >> > them into an InvalidOperationException, e.g like the following:
>
> > >> >> >> > catch(Exception e)
> > >> >> >> >   {
> > >> >> >> >   throw(new InvalidOperationException(string.Format("Caught an
> > >> >> >> > exception: {0}", e.ToString())));
> > >> >> >> >   }
>
> > >> >> >> > I then ran just that test and much to my surprise it succeeded.  
> > >> >> >> > So
> > >> >> >> > then I commented out the try/catch I added and rebuilt and 
> > >> >> >> > retested,
> > >> >> >> > just that test, and again it succeeded.  I tested, individually 
> > >> >> >> > which
> > >> >> >> > a fresh new instance of NUnit GUI, each test that failed and 
> > >> >> >> > they all
> > >> >> >> > succeeded.  For lock I have tests1 - test8.  When I run all lock
> > >> >> >> > tests, test1 - test4 succeed and test5 - test8 fail.  If I run 
> > >> >> >> > test5 -
> > >> >> >> > test8 individually with a fresh instance of NUnit GUI they all
> > >> >> >> > succeed.  The reason for running fresh instances of NUnit GUI is
> > >> >> >> > because it seems once Common gets loaded then all tests from 
> > >> >> >> > them on
> > >> >> >> > will work as expected.  Also, once loading Common fails it will
> > >> >> >> > continue to fail for that instance of NUnit GUI (looks like a 
> > >> >> >> > type
> > >> >> >> > initializer, e.g. static constructor, fails and once that 
> > >> >> >> > happens the
> > >> >> >> > there's no fixing it until the appdomain is unloaded).
>
> > >> >> >> > Thanks,
> > >> >> >> > Nick
>
> > >> >> >> > On Apr 1, 8:16 am, nickdu <[EMAIL PROTECTED]> wrote:
> > >> >> >> >> While we do throw some custom exceptions, my initial guess is 
> > >> >> >> >> that
> > >> >> >> >> this is not what's going on here because some of the tests that 
> > >> >> >> >> are
> > >> >> >> >> failing are positive tests, ones which shouldn't be throwing any
> > >> >> >> >> exceptions.
>
> > >> >> >> >> I just ran another scenario to maybe shed some more light on the
> > >> >> >> >> issue, though it doesn't shed much more light.  I have a class 
> > >> >> >> >> called
> > >> >> >> >> Complete which has a bunch of tests which test our Complete 
> > >> >> >> >> method.
> > >> >> >> >> These tests always work.  I have another class called Lock 
> > >> >> >> >> which tests
> > >> >> >> >> our Lock method.  Lock doesn't work if run by itself (e.g. only
> > >> >> >> >> itself) after just starting the NUnit GUI.  I mentioned 
> > >> >> >> >> originally
> > >> >> >> >> that if I run all the tests these other ones that fail by 
> > >> >> >> >> themselves
> > >> >> >> >> will succeed.  That's not entirely true.  If I start NUnit GUI 
> > >> >> >> >> up
> > >> >> >> >> fresh and then run the Complete tests I can then run the Lock 
> > >> >> >> >> tests,
> > >> >> >> >> using the currently running NUnit GUI that ran the Complete 
> > >> >> >> >> tests,
> > >> >> >> >> successfully.
>
> > >> >> >> >> Here is what my guess was as to what's going on.  It appears the
> > >> >> >> >> tests, and I mean the code in the NUnit test methods, which 
> > >> >> >> >> have code
> > >> >> >> >> which directly references code from the Common assembly, this 
> > >> >> >> >> is the
> > >> >> >> >> assembly that's having the issue of getting loaded, will always 
> > >> >> >> >> work
> > >> >> >> >> whether run separately or when running all tests.  The tests 
> > >> >> >> >> which
> > >> >> >> >> don't reference code from the Common assembly won't work unless 
> > >> >> >> >> the
> > >> >> >> >> Common assembly has already been loaded into the process by 
> > >> >> >> >> first
> > >> >> >> >> running a test that referenced it.  Almost as if the code is 
> > >> >> >> >> getting
> > >> >> >> >> jitted with an app base setting which is correct and allows the 
> > >> >> >> >> .NET
> > >> >> >> >> assembly loader to find the Common assembly using the usual 
> > >> >> >> >> probing
> > >> >> >> >> mechanism, but then executed in a different appdomain which 
> > >> >> >> >> doesn't
> > >> >> >> >> have the correct app base setting such that if/when Common 
> > >> >> >> >> needs to be
> > >> >> >> >> loaded it cannot be found.
>
> > >> >> >> >> Thanks,
> > >> >> >> >> Nick
>
> > >> >> >> >> On Mar 31, 3:15 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
>
> > >> >> >> >> > Hi Nick,
>
> ...
>
> read more »

-- 
You received this message because you are subscribed to the Google Groups 
"NUnit-Discuss" group.
To post to this group, send email to [EMAIL PROTECTED]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/nunit-discuss?hl=en.