[nunit-discuss] Re: Only can run entire suite of tests on 2.6

I did some more tests, and while I found out some new behavior I still
don't know what's going on here.

In order to test your custom exception suggestion I figured I would
take one of the tests that failed and catch all exceptions and turn
them into an InvalidOperationException, e.g like the following:

catch(Exception e)
   {
   throw(new InvalidOperationException(string.Format("Caught an
exception: {0}", e.ToString())));
   }

I then ran just that test and much to my surprise it succeeded.  So
then I commented out the try/catch I added and rebuilt and retested,
just that test, and again it succeeded.  I tested, individually which
a fresh new instance of NUnit GUI, each test that failed and they all
succeeded.  For lock I have tests1 - test8.  When I run all lock
tests, test1 - test4 succeed and test5 - test8 fail.  If I run test5 -
test8 individually with a fresh instance of NUnit GUI they all
succeed.  The reason for running fresh instances of NUnit GUI is
because it seems once Common gets loaded then all tests from them on
will work as expected.  Also, once loading Common fails it will
continue to fail for that instance of NUnit GUI (looks like a type
initializer, e.g. static constructor, fails and once that happens the
there's no fixing it until the appdomain is unloaded).

Thanks,
Nick

On Apr 1, 8:16 am, nickdu <[EMAIL PROTECTED]> wrote:
> While we do throw some custom exceptions, my initial guess is that
> this is not what's going on here because some of the tests that are
> failing are positive tests, ones which shouldn't be throwing any
> exceptions.
>
> I just ran another scenario to maybe shed some more light on the
> issue, though it doesn't shed much more light.  I have a class called
> Complete which has a bunch of tests which test our Complete method.
> These tests always work.  I have another class called Lock which tests
> our Lock method.  Lock doesn't work if run by itself (e.g. only
> itself) after just starting the NUnit GUI.  I mentioned originally
> that if I run all the tests these other ones that fail by themselves
> will succeed.  That's not entirely true.  If I start NUnit GUI up
> fresh and then run the Complete tests I can then run the Lock tests,
> using the currently running NUnit GUI that ran the Complete tests,
> successfully.
>
> Here is what my guess was as to what's going on.  It appears the
> tests, and I mean the code in the NUnit test methods, which have code
> which directly references code from the Common assembly, this is the
> assembly that's having the issue of getting loaded, will always work
> whether run separately or when running all tests.  The tests which
> don't reference code from the Common assembly won't work unless the
> Common assembly has already been loaded into the process by first
> running a test that referenced it.  Almost as if the code is getting
> jitted with an app base setting which is correct and allows the .NET
> assembly loader to find the Common assembly using the usual probing
> mechanism, but then executed in a different appdomain which doesn't
> have the correct app base setting such that if/when Common needs to be
> loaded it cannot be found.
>
> Thanks,
> Nick
>
> On Mar 31, 3:15 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
>
>
>
> > Hi Nick,
>
> > From the error below, it's clear that NUnit itself - rather than your
> > tests - is trying to load the dll in question. This usually happens
> > when a custom exception, defined in one of your assemblies, is
> > uncaught in the tests. Is that a possibility in this scenario?
>
> > If this is the case, then NUnit would  be failing in deserializing the
> > exceptioni because it can't load the assembly in which the exception
> > is defined. One way to find out is to temporarily copy the "missing"
> > assembly to the directory containing nunit-agent.exe, along with any
> > dependencies that may be needed.
>
> > This issue with custom exceptions may or may not be your problem, but
> > it's a long-standing issue and I suggest looking at it next.
>
> > BTW, in NUnit 3.0, the plan is to prevent any exceptions from
> > travelling across the remoting boundary by catching them and
> > transforming them into an xml error report.
>
> > Charlie
>
> > On Sat, Mar 31, 2012 at 6:44 AM, nickdu <[EMAIL PROTECTED]> wrote:
> > > It appears my assumption is somewhat correct about which methods are
> > > failing.  It appears the .NET runtime is having a problem loading
> > > Redbone.Framework.BusinessProcess.Common.dll.  This assembly is in the
> > > same directory as my NUnit test assembly and I have all the other
> > > dependent assemblies there.  This directory is c:\data\development
> > > \Redbone\bpf\tests\bin\debug.  However, when the fusion loader looks
> > > for the Common assembly it's looking in the app base directory where
> > > the NUnit GUI .exe exists. I thought NUnit somehow changed the app
> > > base (maybe by creating another appdomain and loading the tests from
> > > there) to the directory where the test assembly exists?  My main
> > > framework API, which all the tests make use of, gets loaded and that's
> > > in the same directory as the test assembly.  Here is the log from the
> > > fusion log viewer:
>
> > > *** Assembly Binder Log Entry  (3/31/2012 @ 9:36:50 AM) ***
>
> > > The operation failed.
> > > Bind result: hr = 0x80070002. The system cannot find the file
> > > specified.
>
> > > Assembly manager loaded from:  C:\Windows\Microsoft.NET
> > > \Framework64\v4.0.30319\clr.dll
> > > Running under executable  C:\Program Files (x86)\NUnit 2.6\bin\nunit-
> > > agent.exe
> > > --- A detailed error log follows.
>
> > > === Pre-bind state information ===
> > > LOG: User = redbonemobile\nickdu
> > > LOG: DisplayName = Redbone.Framework.BusinessProcess.Common
> > >  (Partial)
> > > WRN: Partial binding information was supplied for an assembly:
> > > WRN: Assembly Name: Redbone.Framework.BusinessProcess.Common | Domain
> > > ID: 1
> > > WRN: A partial bind occurs when only part of the assembly display name
> > > is provided.
> > > WRN: This might result in the binder loading an incorrect assembly.
> > > WRN: It is recommended to provide a fully specified textual identity
> > > for the assembly,
> > > WRN: that consists of the simple name, version, culture, and public
> > > key token.
> > > WRN: See whitepaperhttp://go.microsoft.com/fwlink/?LinkId=109270for
> > > more information and common solutions to this issue.
> > > LOG: Appbase = file:///C:/Program Files (x86)/NUnit 2.6/bin/
> > > LOG: Initial PrivatePath = NULL
> > > LOG: Dynamic Base = NULL
> > > LOG: Cache Base = NULL
> > > LOG: AppName = nunit-agent.exe
> > > Calling assembly : (Unknown).
> > > ===
> > > LOG: This bind starts in default load context.
> > > LOG: Using application configuration file: C:\Program Files
> > > (x86)\NUnit 2.6\bin\nunit-agent.exe.Config
> > > LOG: Using host configuration file:
> > > LOG: Using machine configuration file from C:\Windows\Microsoft.NET
> > > \Framework64\v4.0.30319\config\machine.config.
> > > LOG: Policy not being applied to reference at this time (private,
> > > custom, partial, or location-based assembly bind).
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.DLL.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/lib/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: Attempting download of new URL file:///C:/Program Files (x86)/
> > > NUnit 2.6/bin/addins/Redbone.Framework.BusinessProcess.Common/
> > > Redbone.Framework.BusinessProcess.Common.EXE.
> > > LOG: All probing URLs attempted and failed.
>
> > > Thanks,
> > > Nick
>
> > > On Mar 31, 9:19 am, nickdu <[EMAIL PROTECTED]> wrote:
> > >> Thanks.  I will try that out but I doubt that is the issue.  I'm
> > >> almost positive none of the tests are dependent on the others.  Also,
> > >> in this exception chain somewhere, I guess between the module load
> > >> exception and the serialization exception as I believe those are the
> > >> first and last respectfully, is an exception indicating it can't find
> > >> one of my managed assemblies which is required.  The assembly is
> > >> there, obviously as it works when I run all tests.
>
> > >> By the way, my NUnit test assembly is targeting the .NET 4.0 runtime.
> > >> We also have some unmanaged assemblies brought into the picture by
> > >> some third party managed assemblies.  Not sure if those are causing
> > >> any issues, but as I said everything works as I expect when I run all
> > >> tests.
>
> > >> I think it's some sort of assembly loading issue, not sure if it's
> > >> caused by NUnit or not.  The reason I say this is because of the tests
> > >> that fail.  They appear to be, but I have to do more work here to
> > >> verify, the methods that get past my argument validation code in my
> > >> API and then eventually call into this third party library.
>
> > >> Thanks,
> > >> Nick
>
> > >> On Mar 30, 10:11 pm, Charlie Poole <[EMAIL PROTECTED]> wrote:
>
> > >> > This sounds like it should be a problem with your tests. Some test is
> > >> > required to be run before any others can work. Of course, such a
> > >> > dependency is bad and you would want to root it out.
>
> > >> > To verify that this is the problem, first use NUnit's own tests. You
> > >> > should be able to select and run a subset with no problem.
>
> > >> > If that works, use something like a binary search to find the problem.
> > >> > Turn on checkboxes in the Gui. Select the first test you know of that
> > >> > fails when run alone. See it fail. Now select in addition half of the
> > >> > previously unselected tests above it. If it passes, the failing test
> > >> > depends on one of those you just selected. If it still fails, then the
> > >> > culprit is one of the tests you did not select. Proceed till you find
> > >> > the test which causes the problem.
>
> ...
>
> read more »

-- 
You received this message because you are subscribed to the Google Groups 
"NUnit-Discuss" group.
To post to this group, send email to [EMAIL PROTECTED]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/nunit-discuss?hl=en.