Loading...

nunit-discuss@googlegroups.com

[Prev] Thread [Next]  |  [Prev] Date [Next]

[nunit-discuss] Re: Only can run entire suite of tests on 2.6 nickdu Mon Apr 02 06:08:15 2012

There were trace files created with trace information in them, just
none of it seemed to point to a reason for the issue I'm having.

I don't have any automatic reload setting turned on.

When I mentioned loading it was more about how NUnit loads a class and/
or method in order to execute it.  It seems odd that if I select the
Lock class and run I get different behavior than when I choose each
Lock test method individually and run them.  My guess is that NUnit is
loading the code differently in those two cases.

Thanks,
Nick

On Apr 1, 6:08 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
> Well, the absence of trace info from NUnit at least tells us that NUnit is
> not aware of any problem. I guess we knew that though.
>
> Regarding loading: NUnit does not reload any tests unless you tell it to.
> You should probably turn off any automatic reload settings if you have
> them set. That way you can manually control when a reload takes place.
>
> Charlie
>
>
>
> On Sun, Apr 1, 2012 at 2:55 PM, nickdu <[EMAIL PROTECTED]> wrote:
> > I haven't use NUnit's tracing yet.  I just now tried turning on the
> > most verbose level of tracing.  I looked through all the trace files
> > generated and there didn't seem to be any information in there which
> > pointed to the problem.
>
> > Still doing more testing.  As I mentioned before, if I select the Lock
> > set of tests, and Lock is a class with a bunch of test methods, and
> > run that set, the first four tests, test1 - test4, succeed and the
> > last four, test5 - test8, fail.  The first four are negative test
> > cases which don't make it past the first assembly implementing our API
> > as it validates the parameters and fails.  The last four have valid
> > parameters and thus make it farther into our API and require this
> > Common assembly be loaded.  This is where it fails.
>
> > Interestingly, I can execute test1 successfully and then select test5
> > and execute that successfully in the same instance of the NUnit GUI.
> > However if I select all Lock test cases it fails.  Does NUnit load
> > things differently if you select individual tests version a set of
> > tests by choosing the class?
>
> > Thanks,
> > Nick
>
> > On Apr 1, 1:24 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
> >> Still, this doesn't explain why your tests have anything at all to do
> >> with the primary AppDomain. NUnit does run all tests in a separate
> >> AppDomain and creates a new one each time you reload the tests.
>
> >> Have you turned on NUnit's internal trace and looked at whether it
> >> tells you anything?
>
> >> Charlie
>
> >> On Sun, Apr 1, 2012 at 5:50 AM, nickdu <[EMAIL PROTECTED]> wrote:
> >> > I did some more tests, and while I found out some new behavior I still
> >> > don't know what's going on here.
>
> >> > In order to test your custom exception suggestion I figured I would
> >> > take one of the tests that failed and catch all exceptions and turn
> >> > them into an InvalidOperationException, e.g like the following:
>
> >> > catch(Exception e)
> >> >   {
> >> >   throw(new InvalidOperationException(string.Format("Caught an
> >> > exception: {0}", e.ToString())));
> >> >   }
>
> >> > I then ran just that test and much to my surprise it succeeded.  So
> >> > then I commented out the try/catch I added and rebuilt and retested,
> >> > just that test, and again it succeeded.  I tested, individually which
> >> > a fresh new instance of NUnit GUI, each test that failed and they all
> >> > succeeded.  For lock I have tests1 - test8.  When I run all lock
> >> > tests, test1 - test4 succeed and test5 - test8 fail.  If I run test5 -
> >> > test8 individually with a fresh instance of NUnit GUI they all
> >> > succeed.  The reason for running fresh instances of NUnit GUI is
> >> > because it seems once Common gets loaded then all tests from them on
> >> > will work as expected.  Also, once loading Common fails it will
> >> > continue to fail for that instance of NUnit GUI (looks like a type
> >> > initializer, e.g. static constructor, fails and once that happens the
> >> > there's no fixing it until the appdomain is unloaded).
>
> >> > Thanks,
> >> > Nick
>
> >> > On Apr 1, 8:16 am, nickdu <[EMAIL PROTECTED]> wrote:
> >> >> While we do throw some custom exceptions, my initial guess is that
> >> >> this is not what's going on here because some of the tests that are
> >> >> failing are positive tests, ones which shouldn't be throwing any
> >> >> exceptions.
>
> >> >> I just ran another scenario to maybe shed some more light on the
> >> >> issue, though it doesn't shed much more light.  I have a class called
> >> >> Complete which has a bunch of tests which test our Complete method.
> >> >> These tests always work.  I have another class called Lock which tests
> >> >> our Lock method.  Lock doesn't work if run by itself (e.g. only
> >> >> itself) after just starting the NUnit GUI.  I mentioned originally
> >> >> that if I run all the tests these other ones that fail by themselves
> >> >> will succeed.  That's not entirely true.  If I start NUnit GUI up
> >> >> fresh and then run the Complete tests I can then run the Lock tests,
> >> >> using the currently running NUnit GUI that ran the Complete tests,
> >> >> successfully.
>
> >> >> Here is what my guess was as to what's going on.  It appears the
> >> >> tests, and I mean the code in the NUnit test methods, which have code
> >> >> which directly references code from the Common assembly, this is the
> >> >> assembly that's having the issue of getting loaded, will always work
> >> >> whether run separately or when running all tests.  The tests which
> >> >> don't reference code from the Common assembly won't work unless the
> >> >> Common assembly has already been loaded into the process by first
> >> >> running a test that referenced it.  Almost as if the code is getting
> >> >> jitted with an app base setting which is correct and allows the .NET
> >> >> assembly loader to find the Common assembly using the usual probing
> >> >> mechanism, but then executed in a different appdomain which doesn't
> >> >> have the correct app base setting such that if/when Common needs to be
> >> >> loaded it cannot be found.
>
> >> >> Thanks,
> >> >> Nick
>
> >> >> On Mar 31, 3:15 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
>
> >> >> > Hi Nick,
>
> >> >> > From the error below, it's clear that NUnit itself - rather than your
> >> >> > tests - is trying to load the dll in question. This usually happens
> >> >> > when a custom exception, defined in one of your assemblies, is
> >> >> > uncaught in the tests. Is that a possibility in this scenario?
>
> >> >> > If this is the case, then NUnit would  be failing in deserializing the
> >> >> > exceptioni because it can't load the assembly in which the exception
> >> >> > is defined. One way to find out is to temporarily copy the "missing"
> >> >> > assembly to the directory containing nunit-agent.exe, along with any
> >> >> > dependencies that may be needed.
>
> >> >> > This issue with custom exceptions may or may not be your problem, but
> >> >> > it's a long-standing issue and I suggest looking at it next.
>
> >> >> > BTW, in NUnit 3.0, the plan is to prevent any exceptions from
> >> >> > travelling across the remoting boundary by catching them and
> >> >> > transforming them into an xml error report.
>
> >> >> > Charlie
>
> >> >> > On Sat, Mar 31, 2012 at 6:44 AM, nickdu <[EMAIL PROTECTED]> wrote:
> >> >> > > It appears my assumption is somewhat correct about which methods are
> >> >> > > failing.  It appears the .NET runtime is having a problem loading
> >> >> > > Redbone.Framework.BusinessProcess.Common.dll.  This assembly is in 
> >> >> > > the
> >> >> > > same directory as my NUnit test assembly and I have all the other
> >> >> > > dependent assemblies there.  This directory is c:\data\development
> >> >> > > \Redbone\bpf\tests\bin\debug.  However, when the fusion loader looks
> >> >> > > for the Common assembly it's looking in the app base directory where
> >> >> > > the NUnit GUI .exe exists. I thought NUnit somehow changed the app
> >> >> > > base (maybe by creating another appdomain and loading the tests from
> >> >> > > there) to the directory where the test assembly exists?  My main
> >> >> > > framework API, which all the tests make use of, gets loaded and 
> >> >> > > that's
> >> >> > > in the same directory as the test assembly.  Here is the log from 
> >> >> > > the
> >> >> > > fusion log viewer:
>
> >> >> > > *** Assembly Binder Log Entry  (3/31/2012 @ 9:36:50 AM) ***
>
> >> >> > > The operation failed.
> >> >> > > Bind result: hr = 0x80070002. The system cannot find the file
> >> >> > > specified.
>
> >> >> > > Assembly manager loaded from:  C:\Windows\Microsoft.NET
> >> >> > > \Framework64\v4.0.30319\clr.dll
> >> >> > > Running under executable  C:\Program Files (x86)\NUnit 
> >> >> > > 2.6\bin\nunit-
> >> >> > > agent.exe
> >> >> > > --- A detailed error log follows.
>
> >> >> > > === Pre-bind state information ===
> >> >> > > LOG: User = redbonemobile\nickdu
> >> >> > > LOG: DisplayName = Redbone.Framework.BusinessProcess.Common
> >> >> > >  (Partial)
> >> >> > > WRN: Partial binding information was supplied for an assembly:
> >> >> > > WRN: Assembly Name: Redbone.Framework.BusinessProcess.Common | 
> >> >> > > Domain
> >> >> > > ID: 1
> >> >> > > WRN: A partial bind occurs when only part of the assembly display 
> >> >> > > name
> >> >> > > is provided.
> >> >> > > WRN: This might result in the binder loading an incorrect assembly.
> >> >> > > WRN: It is recommended to provide a fully specified textual identity
> >> >> > > for the assembly,
> >> >> > > WRN: that consists of the simple name, version, culture, and public
> >> >> > > key token.
> >> >> > > WRN: See whitepaperhttp://go.microsoft.com/fwlink/?LinkId=109270for
> >> >> > > more information and common solutions to this issue.
> >> >> > > LOG: Appbase = file:///C:/Program Files (x86)/NUnit 2.6/bin/
> >> >> > > LOG: Initial PrivatePath = NULL
> >> >> > > LOG: Dynamic Base = NULL
> >> >> > > LOG: Cache Base = NULL
> >> >> > > LOG: AppName = nunit-agent.exe
> >> >> > > Calling assembly : (Unknown).
> >> >> > > ===
> >> >> > > LOG: This bind starts in default load context.
> >> >> > > LOG: Using application configuration file: C:\Program Files
> >> >> > > (x86)\NUnit 2.6\bin\nunit-agent.exe.Config
> >> >> > > LOG: Using host configuration file:
> >> >> > > LOG: Using machine configuration file from C:\Windows\Microsoft.NET
> >> >> > > \Framework64\v4.0.30319\config\machine.config.
> >> >> > > LOG: Policy not being applied to reference at this time (private,
> >> >> > > custom, partial, or location-based assembly bind).
> >> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> >> >> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.DLL.
> >> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> >> >> > > NUnit
>
> ...
>
> read more »

-- 
You received this message because you are subscribed to the Google Groups 
"NUnit-Discuss" group.
To post to this group, send email to [EMAIL PROTECTED]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/nunit-discuss?hl=en.