Loading...

nunit-discuss@googlegroups.com

[Prev] Thread [Next]  |  [Prev] Date [Next]

Re: [nunit-discuss] Re: Only can run entire suite of tests on 2.6 Charlie Poole Sun Apr 01 15:00:49 2012

Well, the absence of trace info from NUnit at least tells us that NUnit is
not aware of any problem. I guess we knew that though.

Regarding loading: NUnit does not reload any tests unless you tell it to.
You should probably turn off any automatic reload settings if you have
them set. That way you can manually control when a reload takes place.

Charlie

On Sun, Apr 1, 2012 at 2:55 PM, nickdu <[EMAIL PROTECTED]> wrote:
> I haven't use NUnit's tracing yet.  I just now tried turning on the
> most verbose level of tracing.  I looked through all the trace files
> generated and there didn't seem to be any information in there which
> pointed to the problem.
>
> Still doing more testing.  As I mentioned before, if I select the Lock
> set of tests, and Lock is a class with a bunch of test methods, and
> run that set, the first four tests, test1 - test4, succeed and the
> last four, test5 - test8, fail.  The first four are negative test
> cases which don't make it past the first assembly implementing our API
> as it validates the parameters and fails.  The last four have valid
> parameters and thus make it farther into our API and require this
> Common assembly be loaded.  This is where it fails.
>
> Interestingly, I can execute test1 successfully and then select test5
> and execute that successfully in the same instance of the NUnit GUI.
> However if I select all Lock test cases it fails.  Does NUnit load
> things differently if you select individual tests version a set of
> tests by choosing the class?
>
> Thanks,
> Nick
>
> On Apr 1, 1:24 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
>> Still, this doesn't explain why your tests have anything at all to do
>> with the primary AppDomain. NUnit does run all tests in a separate
>> AppDomain and creates a new one each time you reload the tests.
>>
>> Have you turned on NUnit's internal trace and looked at whether it
>> tells you anything?
>>
>> Charlie
>>
>>
>>
>> On Sun, Apr 1, 2012 at 5:50 AM, nickdu <[EMAIL PROTECTED]> wrote:
>> > I did some more tests, and while I found out some new behavior I still
>> > don't know what's going on here.
>>
>> > In order to test your custom exception suggestion I figured I would
>> > take one of the tests that failed and catch all exceptions and turn
>> > them into an InvalidOperationException, e.g like the following:
>>
>> > catch(Exception e)
>> >   {
>> >   throw(new InvalidOperationException(string.Format("Caught an
>> > exception: {0}", e.ToString())));
>> >   }
>>
>> > I then ran just that test and much to my surprise it succeeded.  So
>> > then I commented out the try/catch I added and rebuilt and retested,
>> > just that test, and again it succeeded.  I tested, individually which
>> > a fresh new instance of NUnit GUI, each test that failed and they all
>> > succeeded.  For lock I have tests1 - test8.  When I run all lock
>> > tests, test1 - test4 succeed and test5 - test8 fail.  If I run test5 -
>> > test8 individually with a fresh instance of NUnit GUI they all
>> > succeed.  The reason for running fresh instances of NUnit GUI is
>> > because it seems once Common gets loaded then all tests from them on
>> > will work as expected.  Also, once loading Common fails it will
>> > continue to fail for that instance of NUnit GUI (looks like a type
>> > initializer, e.g. static constructor, fails and once that happens the
>> > there's no fixing it until the appdomain is unloaded).
>>
>> > Thanks,
>> > Nick
>>
>> > On Apr 1, 8:16 am, nickdu <[EMAIL PROTECTED]> wrote:
>> >> While we do throw some custom exceptions, my initial guess is that
>> >> this is not what's going on here because some of the tests that are
>> >> failing are positive tests, ones which shouldn't be throwing any
>> >> exceptions.
>>
>> >> I just ran another scenario to maybe shed some more light on the
>> >> issue, though it doesn't shed much more light.  I have a class called
>> >> Complete which has a bunch of tests which test our Complete method.
>> >> These tests always work.  I have another class called Lock which tests
>> >> our Lock method.  Lock doesn't work if run by itself (e.g. only
>> >> itself) after just starting the NUnit GUI.  I mentioned originally
>> >> that if I run all the tests these other ones that fail by themselves
>> >> will succeed.  That's not entirely true.  If I start NUnit GUI up
>> >> fresh and then run the Complete tests I can then run the Lock tests,
>> >> using the currently running NUnit GUI that ran the Complete tests,
>> >> successfully.
>>
>> >> Here is what my guess was as to what's going on.  It appears the
>> >> tests, and I mean the code in the NUnit test methods, which have code
>> >> which directly references code from the Common assembly, this is the
>> >> assembly that's having the issue of getting loaded, will always work
>> >> whether run separately or when running all tests.  The tests which
>> >> don't reference code from the Common assembly won't work unless the
>> >> Common assembly has already been loaded into the process by first
>> >> running a test that referenced it.  Almost as if the code is getting
>> >> jitted with an app base setting which is correct and allows the .NET
>> >> assembly loader to find the Common assembly using the usual probing
>> >> mechanism, but then executed in a different appdomain which doesn't
>> >> have the correct app base setting such that if/when Common needs to be
>> >> loaded it cannot be found.
>>
>> >> Thanks,
>> >> Nick
>>
>> >> On Mar 31, 3:15 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
>>
>> >> > Hi Nick,
>>
>> >> > From the error below, it's clear that NUnit itself - rather than your
>> >> > tests - is trying to load the dll in question. This usually happens
>> >> > when a custom exception, defined in one of your assemblies, is
>> >> > uncaught in the tests. Is that a possibility in this scenario?
>>
>> >> > If this is the case, then NUnit would  be failing in deserializing the
>> >> > exceptioni because it can't load the assembly in which the exception
>> >> > is defined. One way to find out is to temporarily copy the "missing"
>> >> > assembly to the directory containing nunit-agent.exe, along with any
>> >> > dependencies that may be needed.
>>
>> >> > This issue with custom exceptions may or may not be your problem, but
>> >> > it's a long-standing issue and I suggest looking at it next.
>>
>> >> > BTW, in NUnit 3.0, the plan is to prevent any exceptions from
>> >> > travelling across the remoting boundary by catching them and
>> >> > transforming them into an xml error report.
>>
>> >> > Charlie
>>
>> >> > On Sat, Mar 31, 2012 at 6:44 AM, nickdu <[EMAIL PROTECTED]> wrote:
>> >> > > It appears my assumption is somewhat correct about which methods are
>> >> > > failing.  It appears the .NET runtime is having a problem loading
>> >> > > Redbone.Framework.BusinessProcess.Common.dll.  This assembly is in the
>> >> > > same directory as my NUnit test assembly and I have all the other
>> >> > > dependent assemblies there.  This directory is c:\data\development
>> >> > > \Redbone\bpf\tests\bin\debug.  However, when the fusion loader looks
>> >> > > for the Common assembly it's looking in the app base directory where
>> >> > > the NUnit GUI .exe exists. I thought NUnit somehow changed the app
>> >> > > base (maybe by creating another appdomain and loading the tests from
>> >> > > there) to the directory where the test assembly exists?  My main
>> >> > > framework API, which all the tests make use of, gets loaded and that's
>> >> > > in the same directory as the test assembly.  Here is the log from the
>> >> > > fusion log viewer:
>>
>> >> > > *** Assembly Binder Log Entry  (3/31/2012 @ 9:36:50 AM) ***
>>
>> >> > > The operation failed.
>> >> > > Bind result: hr = 0x80070002. The system cannot find the file
>> >> > > specified.
>>
>> >> > > Assembly manager loaded from:  C:\Windows\Microsoft.NET
>> >> > > \Framework64\v4.0.30319\clr.dll
>> >> > > Running under executable  C:\Program Files (x86)\NUnit 2.6\bin\nunit-
>> >> > > agent.exe
>> >> > > --- A detailed error log follows.
>>
>> >> > > === Pre-bind state information ===
>> >> > > LOG: User = redbonemobile\nickdu
>> >> > > LOG: DisplayName = Redbone.Framework.BusinessProcess.Common
>> >> > >  (Partial)
>> >> > > WRN: Partial binding information was supplied for an assembly:
>> >> > > WRN: Assembly Name: Redbone.Framework.BusinessProcess.Common | Domain
>> >> > > ID: 1
>> >> > > WRN: A partial bind occurs when only part of the assembly display name
>> >> > > is provided.
>> >> > > WRN: This might result in the binder loading an incorrect assembly.
>> >> > > WRN: It is recommended to provide a fully specified textual identity
>> >> > > for the assembly,
>> >> > > WRN: that consists of the simple name, version, culture, and public
>> >> > > key token.
>> >> > > WRN: See whitepaperhttp://go.microsoft.com/fwlink/?LinkId=109270for
>> >> > > more information and common solutions to this issue.
>> >> > > LOG: Appbase = file:///C:/Program Files (x86)/NUnit 2.6/bin/
>> >> > > LOG: Initial PrivatePath = NULL
>> >> > > LOG: Dynamic Base = NULL
>> >> > > LOG: Cache Base = NULL
>> >> > > LOG: AppName = nunit-agent.exe
>> >> > > Calling assembly : (Unknown).
>> >> > > ===
>> >> > > LOG: This bind starts in default load context.
>> >> > > LOG: Using application configuration file: C:\Program Files
>> >> > > (x86)\NUnit 2.6\bin\nunit-agent.exe.Config
>> >> > > LOG: Using host configuration file:
>> >> > > LOG: Using machine configuration file from C:\Windows\Microsoft.NET
>> >> > > \Framework64\v4.0.30319\config\machine.config.
>> >> > > LOG: Policy not being applied to reference at this time (private,
>> >> > > custom, partial, or location-based assembly bind).
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.DLL.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/
>> >> > > Redbone.Framework.BusinessProcess.Common.DLL.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.DLL.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/
>> >> > > Redbone.Framework.BusinessProcess.Common.DLL.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.DLL.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/
>> >> > > Redbone.Framework.BusinessProcess.Common.DLL.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.EXE.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/
>> >> > > Redbone.Framework.BusinessProcess.Common.EXE.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.EXE.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/
>> >> > > Redbone.Framework.BusinessProcess.Common.EXE.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.EXE.
>> >> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
>> >> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/
>> >> > > Redbone.Framework.BusinessProcess.Common.EXE.
>> >> > > LOG: All probing URLs attempted and failed.
>>
>> >> > > Thanks,
>> >> > > Nick
>>
>> >> > > On Mar 31, 9:19 am, nickdu <[EMAIL PROTECTED]> wrote:
>> >> > >> Thanks.  I will try that out but I doubt that is the issue.  I'm
>> >> > >> almost
>>
>> ...
>>
>> read more »
>
> --
> You received this message because you are subscribed to the Google Groups 
> "NUnit-Discuss" group.
> To post to this group, send email to [EMAIL PROTECTED]
> To unsubscribe from this group, send email to [EMAIL PROTECTED]
> For more options, visit this group at 
> http://groups.google.com/group/nunit-discuss?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"NUnit-Discuss" group.
To post to this group, send email to [EMAIL PROTECTED]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/nunit-discuss?hl=en.