Skip to content

Extract wheels directly from resource #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ivaylo-matov wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Extract wheels directly from resource #9

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0d2db1b
extract wheels directly from resource
ivaylo-matov Nov 3, 2025
338896c
Update DSPythonNet3/DSPythonNet3Evaluator.cs
twastvedt Nov 3, 2025
0176117
validate entry.FullName
ivaylo-matov Nov 3, 2025
2675af4
Merge branch 'Extract-wheels-directly-from-resource_251103_' of https…
ivaylo-matov Nov 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 42 additions & 8 deletions DSPythonNet3/DSPythonNet3Evaluator.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
internal class DynamoCPythonHandleComparer : IEqualityComparer<DynamoCPythonHandle>
{

public bool Equals(DynamoCPythonHandle x, DynamoCPythonHandle y)

Check warning on line 23 in DSPythonNet3/DSPythonNet3Evaluator.cs

View workflow job for this annotation

GitHub Actions / Build and Test 

Nullability of reference types in type of parameter 'y' of 'bool DynamoCPythonHandleComparer.Equals(DynamoCPythonHandle x, DynamoCPythonHandle y)' doesn't match implicitly implemented member 'bool IEqualityComparer<DynamoCPythonHandle>.Equals(DynamoCPythonHandle? x, DynamoCPythonHandle? y)' (possibly because of nullability attributes).

Check warning on line 23 in DSPythonNet3/DSPythonNet3Evaluator.cs

View workflow job for this annotation

GitHub Actions / Build and Test 

Nullability of reference types in type of parameter 'x' of 'bool DynamoCPythonHandleComparer.Equals(DynamoCPythonHandle x, DynamoCPythonHandle y)' doesn't match implicitly implemented member 'bool IEqualityComparer<DynamoCPythonHandle>.Equals(DynamoCPythonHandle? x, DynamoCPythonHandle? y)' (possibly because of nullability attributes).
{
return x.PythonObjectID.Equals(y.PythonObjectID);
}
Expand Down Expand Up @@ -74,7 +74,7 @@
try
{
var pyobj = DSPythonNet3Evaluator.globalScope.Get(PythonObjectID.ToString());
return pyobj.ToString();

Check warning on line 77 in DSPythonNet3/DSPythonNet3Evaluator.cs

View workflow job for this annotation

GitHub Actions / Build and Test 

Possible null reference return.
}
catch (Exception e)
{
Expand Down Expand Up @@ -143,8 +143,8 @@
{ // Session is null when running unit tests.
if (ExecutionEvents.ActiveSession != null)
{
dynamoLogger = ExecutionEvents.ActiveSession.GetParameterValue(ParameterKeys.Logger) as DynamoLogger;

Check warning on line 146 in DSPythonNet3/DSPythonNet3Evaluator.cs

View workflow job for this annotation

GitHub Actions / Build and Test 

Possible null reference assignment.
return dynamoLogger;

Check warning on line 147 in DSPythonNet3/DSPythonNet3Evaluator.cs

View workflow job for this annotation

GitHub Actions / Build and Test 

Possible null reference return.
}
return dynamoLogger;
}
Expand Down Expand Up @@ -370,21 +370,55 @@

Assembly wheelsAssembly = context.LoadFromAssemblyPath(Path.Join(Path.GetDirectoryName(assembly.Location), "DSPythonNet3Wheels.dll"));

string sitePkgsPath = Path.Combine(Python.Included.Installer.EmbeddedPythonHome, "Lib", "site-packages");
Directory.CreateDirectory(sitePkgsPath);

List<string> pipWheelInstall = new List<string>();
await Task.WhenAll(wheelsAssembly.GetManifestResourceNames().Where(x =>

// Extract noo-pip wheels directly from the resource stream
foreach (var resName in wheelsAssembly.GetManifestResourceNames())
{
bool isWheel = x.Contains(".whl");
if (isWheel && x.Contains("pywin32-"))
bool isWheel = resName.EndsWith(".whl");
if (!isWheel) continue;

if (resName.Contains("pywin32-"))
{
pipWheelInstall.Add(x);
return false;
pipWheelInstall.Add(resName);
continue;
}

return isWheel;
}).Select(wheel => Python.Included.Installer.InstallWheel(wheelsAssembly, wheel))).ConfigureAwait(false);
using (var stream = wheelsAssembly.GetManifestResourceStream(resName))
{
if (stream == null || stream.Length == 0)
{
continue;
}

using (var zip = new System.IO.Compression.ZipArchive(stream, System.IO.Compression.ZipArchiveMode.Read, false))
{
foreach (var entry in zip.Entries)
{
if (string.IsNullOrEmpty(entry.Name)) continue;

var destPath = Path.Combine(sitePkgsPath, entry.FullName.Replace('/', Path.DirectorySeparatorChar));
var destDir = Path.GetDirectoryName(destPath);
if (!string.IsNullOrEmpty(destDir))
{
Directory.CreateDirectory(destDir);
}

using (var inStream = entry.Open())
using (var outStream = new FileStream(destPath, FileMode.Create, FileAccess.Write, FileShare.None))
{
await inStream.CopyToAsync(outStream).ConfigureAwait(false);
}
}
}
}
}
Comment on lines 383 to 431
Copy link

Copilot AI Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The foreach loop contains an await statement (line 413), but the loop is not async-aware. This will execute the wheel extractions sequentially rather than in parallel. Consider collecting tasks and using Task.WhenAll() to maintain the parallel execution behavior from the original implementation, which could significantly impact installation performance when multiple wheels need to be extracted.

Copilot uses AI. Check for mistakes.
Copy link
Contributor

@zeusongit zeusongit Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could be problematic?

Copy link
Contributor Author

@ivaylo-matov ivaylo-matov Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this shouldn’t be a problem. Running the extractions sequentially should be safer and still fast enough since we only have around a dozen wheels.
The original version was parallel, but that can sometimes cause disk thrashing or file-exist race conditions. Given that the speed difference is minimal and our tests were already a bit noisy, keeping the extraction sequential felt like a more stable option.
Happy to revert if you think it’s better to restore the parallel behavior.


foreach (var pipWheelResource in pipWheelInstall)
{
{
var pipWheelName = pipWheelResource.Remove(0, "DSPythonNet3Wheels.Resources.".Count());
string wheelPath = Path.Combine(Python.Included.Installer.EmbeddedPythonHome, "Lib", pipWheelName);
using (Stream? stream = wheelsAssembly.GetManifestResourceStream(pipWheelResource))
Expand Down
Loading