[B0TAxy]

Description

This merge request introduces an initial implementation of an NTDS.dit Plugin.
The parser can parse and decrypt secret records (e.g., user hashes, supplemental credentials) from an Active Directory NTDS database.

At this stage, the parser is in a late development preview:

Core functionality for parsing and decrypting secret records is available.

Support for additional record types (e.g., groups, ACLs, domain data) is not yet implemented.

Output support for BloodHound-compatible JSON is planned.

Automated tests have not been added yet — these will be included in a follow-up update.

The goal of creating this MR now is to gather early feedback and design review before finalizing the remaining features.

Checklist

• Implementation (core)
• Tests
• BloodHound export
• Full record support

[joost-j]

Regarding this PR, I've managed to get it to work with my own PR (fox-it/dissect.database#8) by changing a few things:

In the __init__():

ntds_path_key = target.registry.value(
    key="HKLM\\SYSTEM\\CurrentControlSet\\Services\\NTDS\\Parameters", 
    value="DSA Database file"
)
self.ntds = NTDS(target.fs.open(ntds_path_key.value))

Changing all the record[NAME_TO_INTERNAL["XXX"]] references to record.XXX where XXX refers to the official ldapDisplayName, since these names are already automatically converted. Lastly, by changing the database data collection part:

def _collect_pek_and_user_records(self) -> tuple[bytes, Generator[User]]:
    return next(self.ntds.lookup(objectCategory="domainDNS")).pekList, self.ntds.users()

and some other minor adjustments. Even on a small dataset that already seemed to result in a performance boost, since the NTDS class uses indexes to look for the pekList and User records.

Note: not all accounts seem to have entries for dBCSPwd, ntPwdHistory, lmPwdHistory and even unicodePwd, so that probably needs some try/exception wrapping.

Please let me know what you think of this and if you need any help with refactoring & editing the code. You can also DM me on the Dissect discord

[B0TAxy]

Hi @joost-j ,
Thanks for the suggestion! I’ve converted the PR to use your approach, and it’s working well with the changes you mentioned. I’ve also wrapped the optional attributes (dBCSPwd, ntPwdHistory, lmPwdHistory, unicodePwd etc) as you noted. Really appreciate the guidance — the adjustments to record.XXX and the NTDS data collection logic make the code cleaner and improve performance.

[joost-j]

Cool! Good to hear - a quick check from my side also confirms that it works well. When running your ntds.user_accounts plugin I can see the NTDS User class attributes, as well as the hashes, really nice! Hoping to wrap up the NTDS parser this year, so that this PR can be finished as well. Just a small remark; I did need to change the from Cryptodome.Cipher on line 9 to from Crypto.Cipher.

[B0TAxy]

Cool! Good to hear - a quick check from my side also confirms that it works well. When running your ntds.user_accounts plugin I can see the NTDS User class attributes, as well as the hashes, really nice! Hoping to wrap up the NTDS parser this year, so that this PR can be finished as well. Just a small remark; I did need to change the from Cryptodome.Cipher on line 9 to from Crypto.Cipher.

Hi @joost-j , Thanks for the insight! I’ve fixed that import as well.

By the way, do you plan to test your database parser against any older domain controllers? The extraction seems to be a bit different — especially the hash extraction — and I currently don’t have good test data from older DC versions to compare with.

[Schamper]

By the way, do you plan to test your database parser against any older domain controllers? The extraction seems to be a bit different — especially the hash extraction — and I currently don’t have good test data from older DC versions to compare with.

That is the goal, but we will incrementally improve the parser.

[codspeed]

Merging this PR will not alter performance

✅ 11 untouched benchmarks

---

_{Comparing B0TAxy:feautre/ntds_dit_parsing (0c80899) with main (d9a1481)}

[B0TAxy]

By the way, do you plan to test your database parser against any older domain controllers? The extraction seems to be a bit different — especially the hash extraction — and I currently don’t have good test data from older DC versions to compare with.

That is the goal, but we will incrementally improve the parser.

Makes sense. Once you feel the parser is ready for those older versions, could you let me know? I’d be happy to run a few tests on my end and give you some feedback on how it handles the extraction.

[joost-j]

PR fox-it/dissect.database#8 is now ready to be reviewed, and probably will be reviewed somewhere in the upcoming weeks. The PR now also includes most of the decryption routines for PEK lists etc. Once that PR is merged, you can edit your plugin once more to focus mainly on the plugin functionality and record publishing parts, leaving out most of the internals because you can now mostly wrap around those functions from dissect.database. I'll ping you once the PR has been merged!

[Schamper]

I’d be happy to run a few tests on my end and give you some feedback on how it handles the extraction.

If you have any test sets of old domains that are suitable for open-source sharing, that would be greatly appreciated!

[B0TAxy]

PR fox-it/dissect.database#8 is now ready to be reviewed, and probably will be reviewed somewhere in the upcoming weeks. The PR now also includes most of the decryption routines for PEK lists etc. Once that PR is merged, you can edit your plugin once more to focus mainly on the plugin functionality and record publishing parts, leaving out most of the internals because you can now mostly wrap around those functions from dissect.database. I'll ping you once the PR has been merged!

Thanks for the heads-up! I’ve already converted the plugin to use the new functions from your PR, and the code is definitely much cleaner now.

However, I’ve run into a couple of issues with the new routines. First, the hash decryption doesn't seem to be working—it’s returning the same buffer instead of the decrypted result. Second, the tool is now crashing when I try to decrypt the hash for the krbtgt account. Upon early investigation, the buffer I'm getting from unicodePwd for that account is significantly smaller than it used to be.

I'll keep digging into it, but let me know if there are any specific nuances in the new dissect.database functions I should be aware of regarding these cases!

[B0TAxy]

I’d be happy to run a few tests on my end and give you some feedback on how it handles the extraction.

If you have any test sets of old domains that are suitable for open-source sharing, that would be greatly appreciated!

Unfortunately, I don't currently have any test data from old domains that would be suitable for open-source sharing.

[Schamper]

@B0TAxy you need to “unlock” the PEK first now (ntds.pek.unlock(syskey)). Second, it only does the RC4/AES decryption. I meant for the DES decryption to be in dissect.target and re-used from the SAM plugin (it’s currently embedded inside decrypt_single_hash, but it should be separated to a des_decrypt).

If after that you still have some issues, please let me know and if possible provide some test data (PEK, syskey and encrypted blobs).

[B0TAxy]

@B0TAxy you need to “unlock” the PEK first now (ntds.pek.unlock(syskey)). Second, it only does the RC4/AES decryption. I meant for the DES decryption to be in dissect.target and re-used from the SAM plugin (it’s currently embedded inside decrypt_single_hash, but it should be separated to a des_decrypt).

If after that you still have some issues, please let me know and if possible provide some test data (PEK, syskey and encrypted blobs).

Yes, I'm aware of the issue—I had some trouble with Git and it didn't push the new code as expected. That’s fixed now and the latest version is pushed. However, I’m still running into the problem with the krbtgt decryption.

[Schamper]

I can take a look at the fixtures and hash length issue. What exactly do you mean with ground truth data? What would you need?

As for the exception, probably this is because I currently chose to throw an exception when trying to list the groups/members of objects that have been replicated from another DC (phantom objects), as that NTDS won't have that information available. My reasoning was that returning an empty list is not really accurate, since they probably do have groups/members, we just can't know about it. I wanted to experiment with how that would work out in dissect.target, if the exception is annoying (vs a boolean check on is_phantom), we can change that behavior.

[B0TAxy]

I can take a look at the fixtures and hash length issue. What exactly do you mean with ground truth data? What would you need?

As for the exception, probably this is because I currently chose to throw an exception when trying to list the groups/members of objects that have been replicated from another DC (phantom objects), as that NTDS won't have that information available. My reasoning was that returning an empty list is not really accurate, since they probably do have groups/members, we just can't know about it. I wanted to experiment with how that would work out in dissect.target, if the exception is annoying (vs a boolean check on is_phantom), we can change that behavior.

Thanks for looking into the fixtures and hash length!
Regarding ground truth data: I meant that I need the expected cleartext passwords (or target hashes) to verify my parsing logic. For example, @joost-j provided these credentials:

• henk.devries: Winter2025!
• beau.terham: Zomer2027!

Knowing these allows me to confirm that I am extracting the correct hash.
As for the exception: The logic regarding phantom objects makes sense and isn't annoying. However, the error I encountered is actually different. I checked, and it seems to be an IndexError (among others) originating in the backlinks function.

[Schamper]

Except for the changed passwords (which I unfortunately don't remember what I changed them too :)) all data is the same as here: https://github.com/Orange-Cyberdefense/GOAD/blob/main/ad/GOAD/data/config.json

[Schamper]

However, the error I encountered is actually different. I checked, and it seems to be an IndexError (among others) originating in the backlinks function.

I looked into this, and I believe this is a side-effect of the large NTDS.dit probably being "dirty" (can you confirm @joost-j?). Probably this could be fixed if we implement transaction files: fox-it/dissect.database#18

[Schamper]

I looked into this, and I believe this is a side-effect of the large NTDS.dit probably being "dirty" (can you confirm @joost-j?). Probably this could be fixed if we implement transaction files: fox-it/dissect.database#18

We confirmed this is the case and updated the test data in dissect.database. It's now merged too! 🥳 If you could update the file in this PR that'd be appreciated.

[Schamper]

I've made some changes to simplify the plugin further. Depends on fox-it/dissect.database#27 though.

[B0TAxy]

I’ve updated the branch to include more comprehensive tests for the NTDS parsing logic. However, I’ve hit a snag with test_users.

It's currently throwing ValueError: data must be 16 bytes long

Apart from that, the tests should be good now

[Schamper]

It's currently throwing ValueError: data must be 16 bytes long

I've created fox-it/dissect.database#29 to address this inside of the NTDS parser itself. Turns out that it's just a big blob that's stored, we need to split it up into 16 byte chunks ourselves.

[B0TAxy]

It's currently throwing ValueError: data must be 16 bytes long

I've created fox-it/dissect.database#29 to address this inside of the NTDS parser itself. Turns out that it's just a big blob that's stored, we need to split it up into 16 byte chunks ourselves.

Glad to hear you found the fix. Let me know once the PR is merged!

[codecov]

Codecov Report

❌ Patch coverage is 95.58824% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.93%. Comparing base (d9a1481) to head (0c80899).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
dissect/target/plugins/os/windows/ad/ntds.py	96.36%	2 Missing ⚠️
...issect/target/plugins/os/windows/credential/sam.py	90.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1347      +/-   ##
==========================================
+ Coverage   80.91%   80.93%   +0.01%     
==========================================
  Files         396      397       +1     
  Lines       34827    34892      +65     
==========================================
+ Hits        28180    28239      +59     
- Misses       6647     6653       +6     

Flag	Coverage Δ
unittests	80.93% <95.58%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.