Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/ehinkle hip selection #97

Open
wants to merge 41 commits into
base: develop
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
c94f779
copied over CalibHitMerger as starting point for adding workflow
Aug 30, 2023
9916692
this version runs Dummy module
Aug 30, 2023
141d3e2
added separate workflow for dummy
Aug 30, 2023
1cdddcd
committing before reverting dummy script
Aug 30, 2023
9ef00ad
made merge hits run in dummy.py
Aug 30, 2023
8ce2be8
updated product name in yaml
Aug 31, 2023
0e40602
took out the bulk of the merge function
Aug 31, 2023
be9cc8c
added in some comments
Sep 1, 2023
3ea1e8b
renamed files to have better naming scheme
Sep 6, 2023
febd0a1
commented out a bit more of the backtracking
Sep 7, 2023
b8eac60
copied over tracklets macros from module0
Sep 20, 2023
43b9ad9
updated to util path
Sep 20, 2023
9b86a6a
Make tracklet reconstruction module compatible with proto_nd_flow dat…
Oct 2, 2023
6483ea5
Update tracklet reconstruction script to reflect x- and z-coordinate …
Oct 3, 2023
fe63f0d
Fix bug where arrays containing NaNs could be passed to DBSCAN.
Oct 4, 2023
1e89d05
Allow tracklet plotting in proto_nd_flow event display with calib_fin…
Oct 5, 2023
cba5368
Update TrackletReconstruction.yaml to run over charge/calib_prompt_hi…
Oct 5, 2023
e4c9d64
Add capability to plot tracklets dataset created from calib_prompt_hits.
Oct 6, 2023
88b3103
Update distance and charge units for proto_nd_flow tracklet reco script.
Oct 6, 2023
636c313
Change proto_nd_flow tracklet reco yaml to reflect updated charge and…
Oct 6, 2023
5087498
Replace DBSCAN with HDBSCAN in tracklet reconstruction.
Oct 13, 2023
b156bc0
Remove use of RANSAC in proto_nd_flow tracklet reconstruction.
Oct 13, 2023
aa2c586
Add RANSAC and additional run of HDBSCAN back into proto_nd_flow trac…
Oct 16, 2023
5a14bdf
Update HDBSCAN parameters which can be customized in yaml for trackle…
Oct 17, 2023
4929458
Final changes to proto_nd_flow tracklet display notebook.
Oct 19, 2023
77ab323
Merge pull request #81 from DUNE/feature/ehinkle_eventdisplay
edhinkle Oct 19, 2023
a6757b3
Add HIP selection code to proto_nd_flow.
Nov 6, 2023
f51e4fa
Adding ParticleData resource source code in proto_nd_flow and yaml in…
Dec 22, 2023
27ebd60
Adding initial infrastructure for time-dependent version of Disabled …
Dec 22, 2023
f848e97
Allowing for time-based consideration of disabled channels in proto_n…
Jan 4, 2024
3e42379
Adding MC case to disabled_channels resource in proto_nd_flow.
Jan 4, 2024
4969e6d
Adding bug fixes to allow flow to run over Module 1 Bern data.
Jan 5, 2024
6b33390
Updating event display notebook in proto_nd_flow.
Jan 8, 2024
46301e9
Resolve merge conflict between HIP Selection and module 1 reflow reso…
Jan 8, 2024
728f527
Merge pull request #94 from DUNE/feature/ehinkle_module1_reflow_add_r…
edhinkle Jan 8, 2024
cfb29d6
Merging from develop and updating tracklet code and hip selection cod…
Jan 12, 2024
7ad7ce4
Adding additional plotting capabilities for HIP selection.
Jan 14, 2024
8157a43
Updating hit level metrics plotting script for HIP selection.
Jan 15, 2024
845365f
Adding tracklet metrics to HIP selection.
Jan 16, 2024
e01d174
Associating all hits with tracks in HIP selection plotting.
Jan 17, 2024
900dd3e
Update event display, tracklet reconstruction phi angle, and RunData …
Jan 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
added in some comments
Kathryn Sutton committed Sep 1, 2023
commit be9cc8c251399e258f3972cf34a5705ad05163a0
63 changes: 12 additions & 51 deletions src/proto_nd_flow/util/dummy.py
Original file line number Diff line number Diff line change
@@ -10,33 +10,12 @@

class Dummy(H5FlowStage):
'''
Merges the specified cached hits based on their unique channel id and timestamp:
- q -> sum(q)
- ts -> sum(ts * q) / sum(q)

Two algorithms for selecting pairs of hits to merge have been implemented:

- `'pairwise'`: On each iteration, sort all hits by unique y-z position and timestamp. Then, merge every pair of hits that fall within the merge cut. If an odd number of hits fall should be merged, the earliest hit of a group is excluded from the iteration.
- `'last-first'`: On each iteration, sort all hits by unique y-z and timestamp. Then, merge the last pair of hits that fall within the merge cut within each contiguous chunk of neighboring hits.

Both algorithms should produce very similar results.

Example config::

hit_merging:
classname: CalibHitMerger
path: module0_flow.reco.charge.hit_merger
requires:
- 'charge/hits'
params:
events_dset_name: 'charge/events'
hits_name: 'charge/hits'
hit_charge_name: 'charge/hits' # dataset to grab 'q' from
merged_name: 'charge/hits/merged'
mc_hit_frac_dset_name: ``str``, optional, output dataset path for hit charge fraction truth (if present)
merge_cut: 30 # merge hits with delta t < merge_cut [CRS ticks]
merge_mode: 'last-first'
'''
This module was adapted from CalibHitMerger
The goal is to take the charge/calib_prompt_hits and perform a Hough transform
This could also be performed on calib_merged_hits that are in the "final" stage
The outputs are saved as a set of hits along the line that are a subset of the inital input hits
ksutton 8/30/23
'''
class_version = '0.0.0'
defaults = dict(
events_dset_name = 'charge/events',
@@ -81,28 +60,8 @@ def init(self, source_name):
#@staticmethod
def merge_hits(self,hits, weights, seg_fracs, dt_cut, sum_fields=None, weighted_mean_fields=None, max_steps=-1, mode='last-first'):
'''
Combines hits along the second axis on unique channels with a delta t less than dt_cut. Continues
until no hits (or merged hits) are within dt_cut of each other

:param hits: original hits array, shape: (N,M)

:param weights: values used for weighted mean, shape: (N,M)

:param fracs: fractional contributions of true segments per packet

:param dt_cut: delta t cut to merge hits (float) [CRS ticks]

:sum_fields: list of fields in ``hits`` and that should be *summed* when combined, must not be in ``weighted_mean_fields``

:weighted_mean_fields: list of fields in ``hits`` and that should be averaged using the weights when combined, must not be in ``sum_fields``

:param max_steps: optional, maximum number of merges to apply to pairs of neighboring hits (<0 == no limit, 0 == skip merging, >0 == limit steps)

:param mode: optional, merging strategy, either `'last-first'` (on each iteration merges the last hit pair) or `'pairwise'` (on each iteration merges each unique hit pair)

:returns: new hit array, shape: (N,m), new hit charge array, shape: (N,m), and an index array with shape (L,2), [:,0] being the index into the original hit array and [:,1] being the flattened index into the compressed new array

'''
currently does nothing, need to add in Hough transform here
'''

new_seg_bt = np.array(seg_fracs[0])
new_frac_bt = np.array(seg_fracs[1])
@@ -125,16 +84,18 @@ def merge_hits(self,hits, weights, seg_fracs, dt_cut, sum_fields=None, weighted_
def run(self, source_name, source_slice, cache):
super(Dummy, self).run(source_name, source_slice, cache)

#get the event id, backtracking, and hits from the input file
event_id = np.r_[source_slice]
packet_frac_bt = cache['packet_frac_backtrack']
packet_seg_bt = cache['packet_seg_backtrack']
hits = cache[self.hits_name]

#get the new hits, references, and backtracking for the
merged, ref, back_track = self.merge_hits(hits, weights=hits['Q'], seg_fracs=[packet_seg_bt,packet_frac_bt],dt_cut=self.merge_cut, sum_fields=self.sum_fields, weighted_mean_fields=self.weighted_mean_fields, max_steps=self.max_merge_steps, mode=self.merge_mode)

merged_mask = merged.mask['id']
merged_mask = merged.mask['id'] #not sure what this does yet

# first write the new merged hits to the file
# first write the new hits to the file
new_nhit = int((~merged_mask).sum())
merge_slice = self.data_manager.reserve_data(self.merged_name, new_nhit)
merge_idx = np.r_[merge_slice].astype(merged.dtype['id'])