Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert our unit tests and perf harness to use native entry-points #50

Merged
merged 62 commits into from
Jan 26, 2024
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
999f247
Move the delta builder into a separate tool
jkoritzinsky Dec 13, 2023
246ab6e
Add tool to locate CoreCLR
jkoritzinsky Dec 13, 2023
30c64f9
Add basic infra for GoogleTest
jkoritzinsky Dec 13, 2023
2979f9c
Build out some basic structure for locating CoreCLR and discovering f…
jkoritzinsky Dec 19, 2023
54ecd2f
Do a first pass of moving a test over and building out the infrastruc…
jkoritzinsky Dec 20, 2023
1b29d4a
Pass around the raw blobs with file names instead of COM objects.
jkoritzinsky Dec 20, 2023
5eabd67
Get FindAPIs test passing
jkoritzinsky Dec 20, 2023
66b292e
Use new artifacts output support in the SDK to have a simpler layout …
jkoritzinsky Dec 20, 2023
914d4cf
Use DNNE to make a native entrypoint for tests to use for locating ma…
jkoritzinsky Dec 20, 2023
feaa32f
Update CI to run the new tests
jkoritzinsky Dec 20, 2023
575603a
Remove FindAPIs as we have moved this API to the new test harness
jkoritzinsky Dec 20, 2023
1dd1f29
Move ImportAPIs test to the new GTest harness.
jkoritzinsky Dec 21, 2023
75f0551
Add long running import APIs tests to GTest harness.
jkoritzinsky Dec 21, 2023
244ab1c
Split large tests into separate tests per-table
jkoritzinsky Dec 21, 2023
f2c1aa9
Fix test that wasn't using HashByteArray when it should have.
jkoritzinsky Dec 21, 2023
c76044d
Remove indirection tables test into regtest and remove Regression.Uni…
jkoritzinsky Dec 22, 2023
02d6326
Revert splitting the big tests into smaller tests. This makes the CTe…
jkoritzinsky Dec 22, 2023
6d91926
Remove unused globals
jkoritzinsky Dec 22, 2023
edfa010
Start converting regnative to an native exe-based perf benchmark
jkoritzinsky Dec 22, 2023
318b642
Move benchmark suite to be driven by google/benchmark in native code.
jkoritzinsky Dec 22, 2023
0524a01
Lazily read in/generate metadata files and pass around spans of the f…
jkoritzinsky Dec 22, 2023
f0ffaa0
Use WIL and lookup the .NET Framework install location in regtest, no…
jkoritzinsky Dec 22, 2023
2fd13f6
Manually produce a minimal image with indirection tables at runtime u…
jkoritzinsky Dec 22, 2023
e327198
Reference winhdrs from regpal
jkoritzinsky Dec 22, 2023
a4f5625
Add dncp include, define iids in pal.cpp for tests, increase discover…
jkoritzinsky Dec 25, 2023
56a49c4
Add includes for standard type definitions
jkoritzinsky Dec 25, 2023
7f43647
Fix current errors in unix/macos builds
jkoritzinsky Dec 27, 2023
58a0eee
DNNE doesn't add the lib prefix on non-Windows
jkoritzinsky Dec 27, 2023
b9c07d9
Set define to avoid windows.h include in corsym.h
jkoritzinsky Dec 27, 2023
c8b0db7
Fix build on Windows
jkoritzinsky Dec 27, 2023
4c1bb24
Fix references to IMAGE_DEBUG_DIRECTORY in regtest and use the define…
jkoritzinsky Dec 27, 2023
fa3baba
Add THROW_IF_FAILED macro for non-Windows and increase test discovery…
jkoritzinsky Dec 27, 2023
92b28d4
These CI machines are slow
jkoritzinsky Dec 27, 2023
7fdc511
Polyfill HRESULT asserts that gtest doesn't provide on non-Windows
jkoritzinsky Dec 27, 2023
16d596b
Add some logging because discovery shouldn't take this long
jkoritzinsky Dec 27, 2023
10eccbf
Remove unused function
jkoritzinsky Dec 27, 2023
93480b2
Add more logging to see why discovery is taking a long time
jkoritzinsky Dec 27, 2023
e0251dc
Add more diagnostics and fix path for Unix platforms
jkoritzinsky Dec 27, 2023
c074d07
Significantly increase discovery time
jkoritzinsky Dec 27, 2023
3e8210d
Set no-config imported locations and correctly handle reading R2Rd im…
jkoritzinsky Dec 27, 2023
0bca273
Merge branch 'no-dotnet-test' of /mnt/e/source/DNMD into no-dotnet-test
jkoritzinsky Dec 27, 2023
ec4089c
Don't validate the value of the reserved out parameter (as its value …
jkoritzinsky Dec 27, 2023
83ec060
Add name overrides to avoid duplicate names for the corelib-only tests.
jkoritzinsky Dec 27, 2023
9b2215d
Don't install testing dependencies
jkoritzinsky Dec 27, 2023
0dcbfce
index on no-dotnet-test: 9b2215d Don't install testing dependencies
jkoritzinsky Dec 28, 2023
902eb68
On no-dotnet-test: Use nethost instead of a managed helper
jkoritzinsky Dec 28, 2023
e391dd4
Fail on any build failure
jkoritzinsky Dec 28, 2023
ec44019
Use nethost to find coreclr without starting up the runtime
jkoritzinsky Dec 28, 2023
8a82c12
Initialize the runtime on non-Windows
jkoritzinsky Dec 28, 2023
a889abf
Remove resolved TODO
jkoritzinsky Dec 28, 2023
a7a53c4
Set env var to suppress first-time-use message
jkoritzinsky Dec 28, 2023
a2abedd
Run dotnet --info for diagnostics
jkoritzinsky Dec 28, 2023
cc16629
Dump more diagnostic
jkoritzinsky Dec 28, 2023
b94f4db
Move the diagnostic
jkoritzinsky Dec 28, 2023
faba662
Add more logging
jkoritzinsky Dec 29, 2023
dd45d85
Dump hostfxr path
jkoritzinsky Dec 29, 2023
2fd064e
Specify dotnet root based on hostfxr that is discovered.
jkoritzinsky Dec 29, 2023
05857f1
Remove unused locator project now that we're using nethost.
jkoritzinsky Dec 29, 2023
25543f3
Remove configuration limitations, we don't need them any more as we a…
jkoritzinsky Dec 29, 2023
afe73e3
Apply suggestions from code review
jkoritzinsky Jan 2, 2024
0445092
PR feedback
jkoritzinsky Jan 26, 2024
9f68f49
Don't crack open files for discovery. Use a heuristic instead
jkoritzinsky Jan 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@ jobs:
with:
dotnet-version: '8.0.x'
dotnet-quality: 'preview'

- name: List dotnet information
run: dotnet --info

- name: Build Vendored Dependencies
if: ${{ !matrix.use-vendored-libs }}
Expand All @@ -46,13 +49,8 @@ jobs:
cmake -S . -B artifacts -DCMAKE_BUILD_TYPE=${{ matrix.flavor }} -DINCLUDE_VENDORED_LIBS=${{ matrix.use-vendored-libs }}
cmake --build artifacts --config ${{ matrix.flavor }} --target install

- name: Build Managed Test Components
run: dotnet build --configuration ${{ matrix.flavor }}
working-directory: ./test

- name: Run Unit Tests
run: dotnet test --configuration ${{ matrix.flavor }} --logger trx --results-directory "TestResults-${{ matrix.os }}-${{ matrix.flavor }}"
working-directory: ./test/Regression.UnitTests
- name: Run Tests
run: ctest --test-dir artifacts --output-on-failure -C ${{ matrix.flavor }}

# - name: Upload Test Results
# if: always()
Expand Down
3 changes: 3 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,8 @@ include_directories(src/inc)
include_directories(src/inc/external) # Hiding the "external" subdirectory due to uses of <...> in cor.h.

add_subdirectory(src/)

enable_testing()

add_subdirectory(test/)

16 changes: 3 additions & 13 deletions Directory.Build.props
Original file line number Diff line number Diff line change
@@ -1,22 +1,12 @@
<Project>
<PropertyGroup>
<Configuration Condition="'$(Configuration)' == ''">Debug</Configuration>
</PropertyGroup>

<PropertyGroup>
<RepoDir>$(MSBuildThisFileDirectory)</RepoDir>
<BaseArtifactsPath>$(RepoDir)artifacts</BaseArtifactsPath>
<ArtifactsDir>$(BaseArtifactsPath)/$(Configuration)/</ArtifactsDir>

<BaseIntermediateOutputPath>$(ArtifactsDir)obj/$(MSBuildProjectName)</BaseIntermediateOutputPath>
<BaseOutputPath>$(ArtifactsDir)bin/$(MSBuildProjectName)</BaseOutputPath>
<UseArtifactsOutput>true</UseArtifactsOutput>
<ArtifactsPath>$(MSBuildThisFileDirectory)/artifacts/managed</ArtifactsPath>

<!-- Define the intermediate and output paths so the configuration isn't appended -->
<IntermediateOutputPath>$(BaseIntermediateOutputPath)</IntermediateOutputPath>
<OutputPath>$(BaseOutputPath)</OutputPath>
<NativeOutputPath Condition="$([MSBuild]::IsOSPlatform('Windows'))">$(BaseArtifactsPath)/bin</NativeOutputPath>
<NativeOutputPath Condition="!$([MSBuild]::IsOSPlatform('Windows'))">$(BaseArtifactsPath)/lib</NativeOutputPath>

<TargetFrameworkLatest>net8.0</TargetFrameworkLatest>
<TargetFramework>net8.0</TargetFramework>
</PropertyGroup>
</Project>
7 changes: 7 additions & 0 deletions src/inc/internal/dnmd_platform.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@

#endif // !BUILD_WINDOWS

// Machine code masks for native (R2R) images
jkoritzinsky marked this conversation as resolved.
Show resolved Hide resolved
#define IMAGE_FILE_MACHINE_OS_MASK_APPLE 0x4644
#define IMAGE_FILE_MACHINE_OS_MASK_FREEBSD 0xADC4
#define IMAGE_FILE_MACHINE_OS_MASK_LINUX 0x7B79
#define IMAGE_FILE_MACHINE_OS_MASK_NETBSD 0x1993
#define IMAGE_FILE_MACHINE_OS_MASK_SUN 0x1992

#include <cstdlib>
#include <cstdint>

Expand Down
40 changes: 36 additions & 4 deletions src/inc/internal/dnmd_tools_platform.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
#include <fstream>
#include <stdexcept>
#include <array>
#include <cstring>

#include "dnmd_platform.hpp"
#include "span.hpp"
Expand Down Expand Up @@ -85,6 +86,33 @@ inline bool write_out_file(char const* file, malloc_span<uint8_t> b)
return true;
}

inline bool find_pe_image_bitness(uint16_t machine, uint8_t& bitness)
{
#define MAKE_MACHINE_CASE(x) \
case ((x) ^ IMAGE_FILE_MACHINE_OS_MASK_APPLE): \
case ((x) ^ IMAGE_FILE_MACHINE_OS_MASK_FREEBSD): \
case ((x) ^ IMAGE_FILE_MACHINE_OS_MASK_LINUX): \
case ((x) ^ IMAGE_FILE_MACHINE_OS_MASK_NETBSD): \
case ((x) ^ IMAGE_FILE_MACHINE_OS_MASK_SUN): \
case (x)

switch (machine)
{
MAKE_MACHINE_CASE(IMAGE_FILE_MACHINE_I386):
MAKE_MACHINE_CASE(IMAGE_FILE_MACHINE_ARM):
bitness = 32;
return true;
MAKE_MACHINE_CASE(IMAGE_FILE_MACHINE_AMD64):
MAKE_MACHINE_CASE(IMAGE_FILE_MACHINE_ARM64):
bitness = 64;
return true;
default:
return false;
}

#undef MAKE_MACHINE_CASE
}

inline bool get_metadata_from_pe(malloc_span<uint8_t>& b)
{
if (b.size() < sizeof(IMAGE_DOS_HEADER))
Expand All @@ -111,8 +139,13 @@ inline bool get_metadata_from_pe(malloc_span<uint8_t>& b)
uint16_t section_header_count;
uint8_t* section_header_begin;
auto nt_header_any = (PIMAGE_NT_HEADERS)(b + dos_header->e_lfanew);
if (nt_header_any->FileHeader.Machine == IMAGE_FILE_MACHINE_AMD64
|| nt_header_any->FileHeader.Machine == IMAGE_FILE_MACHINE_ARM64)
uint16_t machine = nt_header_any->FileHeader.Machine;

uint8_t bitness;
if (!find_pe_image_bitness(machine, bitness))
return false;

if (bitness == 64)
{
auto nt_header64 = (PIMAGE_NT_HEADERS64)nt_header_any;
if (remaining_pe_size < sizeof(*nt_header64))
Expand All @@ -122,8 +155,7 @@ inline bool get_metadata_from_pe(malloc_span<uint8_t>& b)
section_header_begin = (uint8_t*)&nt_header64[1];
dotnet_dir = &nt_header64->OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR];
}
else if (nt_header_any->FileHeader.Machine == IMAGE_FILE_MACHINE_I386
|| nt_header_any->FileHeader.Machine == IMAGE_FILE_MACHINE_ARM)
else if (bitness == 32)
{
auto nt_header32 = (PIMAGE_NT_HEADERS32)nt_header_any;
if (remaining_pe_size < sizeof(*nt_header32))
Expand Down
2 changes: 1 addition & 1 deletion src/interfaces/metadataimport.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2727,7 +2727,7 @@ HRESULT STDMETHODCALLTYPE MetadataImportRO::GetCustomAttributeByName(
mdcursor_t cursor;
uint32_t count;
if (!md_create_cursor(_md_ptr.get(), mdtid_CustomAttribute, &cursor, &count))
return CLDB_E_RECORD_NOTFOUND;
return S_FALSE; // If no custom attributes are defined, treat it the same as if the attribute is not found.

char buffer[1024];
pal::StringConvert<WCHAR, char> cvt{ szName, buffer };
Expand Down
35 changes: 34 additions & 1 deletion test/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,37 @@
# Configure the compiler
include(../configure.cmake)
include(FindNetHost.cmake)

add_subdirectory(regnative/)
if (POLICY CMP0135)
cmake_policy(SET CMP0135 NEW) # Set timestamps in downloaded archives to the time of download.
endif()

include(FetchContent)
FetchContent_Declare(
googletest
URL https://github.com/google/googletest/archive/03597a01ee50ed33e9dfd640b249b4be3799d395.zip
jkoritzinsky marked this conversation as resolved.
Show resolved Hide resolved
)
# For Windows: Prevent overriding the parent project's compiler/linker settings
set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
set(INSTALL_GTEST OFF CACHE BOOL "" FORCE)

FetchContent_Declare(
benchmark
GIT_REPOSITORY
https://github.com/google/benchmark.git
GIT_TAG
v1.8.3
)

# Don't build the tests for the benchmark library.
set(BENCHMARK_ENABLE_TESTING OFF CACHE BOOL "" FORCE)
set(BENCHMARK_ENABLE_INSTALL OFF CACHE BOOL "" FORCE)
FetchContent_MakeAvailable(googletest benchmark)

include(GoogleTest)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
add_subdirectory(regpal)
add_subdirectory(regperf)
add_subdirectory(regtest)
12 changes: 0 additions & 12 deletions test/DNMD.Tests.sln
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@ Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 17
VisualStudioVersion = 17.5.33026.144
MinimumVisualStudioVersion = 10.0.40219.1
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Regression.UnitTests", "Regression.UnitTests\Regression.UnitTests.csproj", "{D5105B2A-6016-4B1C-A46B-5A841AE5AD26}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Regression.Performance", "Regression.Performance\Regression.Performance.csproj", "{3ED137D0-B5A2-4309-BDA5-BD9DE3FFC752}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Regression.TargetAssembly", "Regression.TargetAssembly\Regression.TargetAssembly.ilproj", "{D3BEEF3D-C137-49AC-96DD-E5B93E2F4C21}"
EndProject
Global
Expand All @@ -15,14 +11,6 @@ Global
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{D5105B2A-6016-4B1C-A46B-5A841AE5AD26}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D5105B2A-6016-4B1C-A46B-5A841AE5AD26}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D5105B2A-6016-4B1C-A46B-5A841AE5AD26}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D5105B2A-6016-4B1C-A46B-5A841AE5AD26}.Release|Any CPU.Build.0 = Release|Any CPU
{3ED137D0-B5A2-4309-BDA5-BD9DE3FFC752}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{3ED137D0-B5A2-4309-BDA5-BD9DE3FFC752}.Debug|Any CPU.Build.0 = Debug|Any CPU
{3ED137D0-B5A2-4309-BDA5-BD9DE3FFC752}.Release|Any CPU.ActiveCfg = Release|Any CPU
{3ED137D0-B5A2-4309-BDA5-BD9DE3FFC752}.Release|Any CPU.Build.0 = Release|Any CPU
{D3BEEF3D-C137-49AC-96DD-E5B93E2F4C21}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D3BEEF3D-C137-49AC-96DD-E5B93E2F4C21}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D3BEEF3D-C137-49AC-96DD-E5B93E2F4C21}.Release|Any CPU.ActiveCfg = Release|Any CPU
Expand Down
22 changes: 22 additions & 0 deletions test/FindNetHost.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
execute_process(
COMMAND ${CMAKE_COMMAND} -E env DOTNET_NOLOGO=1 dotnet msbuild FindNetHostDir.proj -t:OutputNetHostDir -nologo
OUTPUT_VARIABLE NET_HOST_DIR
OUTPUT_STRIP_TRAILING_WHITESPACE
COMMAND_ERROR_IS_FATAL ANY
WORKING_DIRECTORY ${CMAKE_CURRENT_LIST_DIR})

string(STRIP ${NET_HOST_DIR} NET_HOST_DIR)

if (WIN32)
add_library(nethost IMPORTED SHARED)
set_target_properties(nethost PROPERTIES
IMPORTED_LOCATION ${NET_HOST_DIR}/nethost.dll
IMPORTED_IMPLIB ${NET_HOST_DIR}/nethost.lib)
else()
add_library(nethost IMPORTED STATIC)
target_compile_definitions(nethost INTERFACE NETHOST_USE_AS_STATIC)
set_target_properties(nethost PROPERTIES
IMPORTED_LOCATION ${NET_HOST_DIR}/libnethost.a)
endif()

target_include_directories(nethost INTERFACE ${NET_HOST_DIR})
8 changes: 8 additions & 0 deletions test/FindNetHostDir.proj
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<Project Sdk="Microsoft.NET.Sdk" DefaultTargets="OutputNetHostDir">
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
jkoritzinsky marked this conversation as resolved.
Show resolved Hide resolved
</PropertyGroup>
<Target Name="OutputNetHostDir">
<Message Text="$(NetCoreTargetingPackRoot)/Microsoft.NETCore.App.Host.$(NETCoreSdkRuntimeIdentifier)/$(BundledNETCoreAppPackageVersion)/runtimes/$(NETCoreSdkRuntimeIdentifier)/native" Importance="high" />
</Target>
</Project>
112 changes: 0 additions & 112 deletions test/Regression.Performance/Program.cs

This file was deleted.

21 changes: 0 additions & 21 deletions test/Regression.Performance/Regression.Performance.csproj

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
<Project Sdk="Microsoft.NET.Sdk.IL">
<PropertyGroup>
<TargetFramework>$(TargetFrameworkLatest)</TargetFramework>
<OutputType>Library</OutputType>
<EnableDefaultCompileItems>false</EnableDefaultCompileItems>
</PropertyGroup>
Expand Down
Loading
Loading