How to access package references within a Task for

Is it possible to use package references like Microsoft.CodeAnalysis within a Microsoft Build Task running with dotnet build?

I'm using netstandard1.5 for the task library and the following package references:

<PackageReference Include="Microsoft.Build" Version="15.3.0-preview-000117-01" />
<PackageReference Include="Microsoft.Build.Utilities.Core" Version="15.3.0-preview-000117-01" />
<PackageReference Include="Microsoft.CodeAnalysis.Common" Version="2.2.0" />

The consuming project is a netcoreapp1.1. The task fails with an

The "MyTask" task failed unexpectedly. System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.CodeAnalysis, Version=2.2.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified. File name: 'Microsoft.CodeAnalysis, Version=2.2.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'

exception.

When trying to load the package reference via AssemblyLoadContext.Default.LoadFromAssemblyPath I get

Could not load file or assembly 'Microsoft.CodeAnalysis, Version=2.2.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.

without any additional information. This does not work when running in a .net core console application either. I tried a second (simpler) package, which can be loaded via LoadFromAssemblyPath in a .net core console application, but still it does not work when running as dotnet build Task.

Both assemblies can be loaded when running in classical net4.6 with Assembly.LoadFile in a console application and when running as Task though.

Unfortunately task dlls cannot reference any packages - they are loaded into the running msbuild instance and will only be able to reference the dlls that are available to the hosting msbuild instance. Any other DLLs could be loaded via AssemblyLoadContext in .net core or AppDomain.Load() in .net framework. No dependency resolution is performed for the task's dll file, this also means that there may be conflicts with already loaded dlls, different parts of msbuild or other tasks. In .NET Framework MSBuild, tasks can be isolated to their own AppDomains, but this mechanism is not available on .NET Core.