MS Build Error on SqlPersistenceTask

igobl · September 27, 2023, 10:25am

Hi,

I am looking for some help with an msbuild error we are experiencing as seen below.

Our current set up is we have the following projects all written in .net standard 2.0 (exact project names changed):

Setup - A NuGet package that is used to carry out some common setup such as defining conventions, transport, routing etc.
Contract - Contains POCOs for messages
Core - Contains message handlers

We were previously hosting this setup in a .NET framework 4.6.2 web app. However we are now creating a .NET 6 web app that will host these projects instead. The projects mentioned above will remain .NET standard for the moment, just the host will be .NET 6.

When we try to build using DotNetCoreCLI@2 we get the below error. I’m pretty sure I can see why this is failing, the task is looking for a dll in the wrong folder, the dll is published to Release/net6/.

##[error]SqlPersistenceTask: AssemblyPath ‘Host/obj/Release/netstandard2.0/Core.dll’ does not exist.

I was hoping someone here could hopefully shed some more light on what this task is doing or point me to some documentation that might give me some more details so I can continue debugging? I have looked in github and on this forum and couldn’t see any outstanding issues around this, so I would say it is likely that the issue is on my side somewhere

All projects mentioned are referencing NServiceBus v7.8.2.

bording · September 27, 2023, 9:32pm

Hi @igobl,

Can you share more details about the relationship between all of these projects and what NServiceBus packages all of them are referencing?

In other words, please share all ProjectReference and PackageReference entries in your .csproj files.

Some other things that would be helpful to know:

What is the exact syntax you are passing to the DotNetCoreCLI@2 task?
Do you see any problems when you build locally with dotnet build?

igobl · September 28, 2023, 1:46pm

Hi @bording, sure thing.

Hopefully the below has enough detail

So below is the pipeline step that we are running, a pretty straight forward dotnet build. We have the same step running on some of our other apps (where all projects in this app are .net 6) and they build with the same Messaging packages without any issues. Also the projects build fine locally, both in visual studio in Debug/Release as well as using dotnet build. The only place we see this issue is in the pipeline.

      - task: DotNetCoreCLI@2
        displayName: 'Build ${{ variables.projectDir }} Projects'
        inputs:
          command: 'build'
          projects: '${{ variables.projectDir }}/App.Host.csproj'
          arguments: '--configuration Release'

App.Host
Our .net 6 web app host.

  <PackageReference Include="Messaging.Contract" Version="1.4.0" />
  <PackageReference Include="Messaging.Core" Version="1.13.2" />
  <PackageReference Include="Messaging.Extensions" Version="1.4.2" />
  <PackageReference Include="NServiceBus" Version="7.8.2" />

  <ProjectReference Include="..\App.Core\App.Core.csproj" />
  <ProjectReference Include="..\App.Contract\App.Contract.csproj" />

App.Core
App Core project, containing business logic, including message handlers. netstandard2.0

  <PackageReference Include="Messaging.Contract" Version="1.4.0" />
  <PackageReference Include="Messaging.Core" Version="1.13.2" />
  <PackageReference Include="NServiceBus" Version="7.8.2" />

<ProjectReference Include="..\App.Contract\App.Contract.csproj" />

App.Contract
App Contract containing POCO for Commands, Events, Queries etc. netstandard2.0

<PackageReference Include="Messaging.Contract" Version="0.4.1" />

Messaging.Core - Nuget Package
Project published to package feed that holds recurring functionality used in our system when configuring and starting endpoints. netstandard2.0

  <PackageReference Include="NServiceBus" Version="7.8.2" />
  <PackageReference Include="NServiceBus.Extensions.DependencyInjection" Version="1.0.1" />
  <PackageReference Include="NServiceBus.Metrics.ServiceControl" Version="3.0.6" />
  <PackageReference Include="NServiceBus.Newtonsoft.Json" Version="2.3.0" />
  <PackageReference Include="NServiceBus.Persistence.Sql" Version="6.5.1" />
  <PackageReference Include="NServiceBus.SagaAudit" Version="3.0.1" />
  <PackageReference Include="NServiceBus.RabbitMQ" Version="6.1.1" />
  <PackageReference Include="NServiceBus.Transport.AzureServiceBus" Version="2.0.2" />

  <ProjectReference Include="..\Messaging.Contract\Messaging.Contract.csproj" />
  <ProjectReference Include="..\Messaging.Extensions\Messaging.Extensions.csproj" />

Messaging.Extensions - Nuget Package
Project published to package feed and contains custom behaviours that we may/may not include in our apps. netstandard2.0

    <PackageReference Include="NServiceBus" Version="7.8.2" />
    <PackageReference Include="NServiceBus.Extensions.DependencyInjection" Version="1.0.1" />
    <PackageReference Include="NServiceBus.Metrics.ServiceControl" Version="3.0.6" />
    <PackageReference Include="NServiceBus.Newtonsoft.Json" Version="2.3.0" />
    <PackageReference Include="NServiceBus.Persistence.Sql" Version="6.5.1" />
    <PackageReference Include="NServiceBus.SagaAudit" Version="3.0.1" />
    <PackageReference Include="NServiceBus.RabbitMQ" Version="6.1.1" />
    <PackageReference Include="NServiceBus.Transport.AzureServiceBus" Version="2.0.2" />
    
    <ProjectReference Include="..\Messaging.Contract\Messaging.Contract.csproj" />

Messaging.Contract - Nuget Package
Project published to package feed, contains some attributes/interfaces that we use in conventions. netstandard2.0

 <PackageReference Include="NServiceBus" Version="7.8.2" />

igobl · September 29, 2023, 11:57am

@bording FYI I managed to fix the build by disabling script generation using the tag found here Controlling script generation • Sql Persistence • Particular Docs.

<SqlPersistenceGenerateScripts>false</SqlPersistenceGenerateScripts>

I think I need to do some more reading of the NSB docs around exactly what those scripts are doing and how they are being executed. From what I could see, the scripts were expecting the obj folder to contain App.Core.dll. But as we were building App.Host which was referencing App.Core, the App.Core.dll would not be contained in the obj folder when built in release mode. I’m still not fully sure why this only shows up in Azure DevOps, as I would have expected it to fail using local msbuild in the same way.

That is my current understanding, but I need to do more reading in the area to fully understand what was happening and the implications of disabling those scripts.

bording · October 9, 2023, 11:42pm

@igobl Yes, turning off the script generation would make the error stop happening because the error is a script generation error.

The scripts are used in two ways:

If you have installers enabled, those scripts are what the persister will execute to create the tables needed to operate
If you don’t use installers, the scripts can be manually run to create the required tables instead.

Having those scripts can be important to ensure the endpoints can run properly.

It’s still not clear to me why you would be getting the error that you’re seeing. At this point I think I would need to see an actual solution to try and figure out what is going on. If you could create a simplified repro solution that demonstrates the problem, that would be ideal.

Another possible option would be to capture an MSBuild binlog from the CI run that is having the failure. If that would be something you could do, it would be a good idea to open a support case and send the binlog there because binlogs can have data like environment variables that you might not want to share publicly.