====== NPAC Internal Weekly Notes ====== * [[https://pitt.zoom.us/my/foranw|Zoom]] * Prev (20241107 - 20250820) [[https://tetra-argo.atlassian.net/wiki/spaces/M/pages/56819713/Weekly|tetra confluence]] | Elijah | 50% | | Will | 30% | | Andrew | 10% | Format Notes: ''-'' enumerate list w/''~ 1h'' time spend ''*'' unordered bullet w/ ''| 1h'' time estimate ideally: link to git diff (''compare/main~3..main'') & link to wiki page. ===== 2026-06-03 ===== * undergrand BME for NM * Bea and Finn * * bids2nda: E Lorenc (brown), Yarik (Dartmouth) * https://etherpad.opendev.org/p/bids2nda * case for python: bids library, nda-tools, uvx/pip install * MRSI -- chan or andrewR (Finn) * * phanom paired with sesssion? (inhomo confound) * rfpulse in empty spectrum. ERCT * redcap <-> xnat incidental finding * api - small scripts: connect, pull, push (id=0 + force=True is new) * need to review fields (instruments) * survey links. then XNAT * ppt to questions * Jira * protocol QC * exclude seqno > 100 * series number for same sequence name within session (color if before or after error; matches template) * xnat tagging * MRSI: gam adjust; * line width (fwhm) / SNR from LCModel? use matching cs.coord FWHM=.... * grep FWHM LCModel/v2idxfix/*/csi.coord * L/R Cad. Thal * Cr for gam adjusted * Samira access to rhea for coords? * https://pubs.rsna.org/doi/10.1148/radiol.2402050314 * neurodesk: containers for OpenRecon; BART * mallory on rhea/npac@cerebro2. ask nathan oac_ix_luna ===== 2026-04-22 ===== * [[neuromelanin]] [[:npac:mp2rage]] * phillips using whole brain & not CC (most stable voxel in residual?) for reference * brainlife.io * install VMs * create/revive xnat connector * build bids curator ? * port file-curator alert? * new tSequenceFileName in DB; can parse exar1 files exported from scanner. * https://github.com/NPACore/mrrc-hdr-qa/commit/ff31712ea02b4abcb2fad52fa57a33026d65e861 * https://gitlab.com/tobiasrautenkranz/exar1-read/ ===== 2026-04-15 ===== AP * investigating SR edits to NM Toolbox, only two changes were made. Unfortunately, there is no QC to check. [[:npac:neuromelanin]] * Changes are addition of antsMultivariateTemplateConstruction2 (builds 'unbiased' population template) & custom masks were used * Found QC for some runs of the NM Toolbox that we can look over on /eru. * Apparently there are still some issues with the modified pipeline (bright spots, have not seed this). Deepak had a thought to try MATLAB computer vision techniques, similar to microscopy segmentation, to delineate the boundaries of the ROIs. * How precise is estimation of motion from MRI, and would a gyroscope really improve this? Or is the internal motion estimation sufficient but not optimized yet? Things to consider, warping due to magnetic field inhomogeneity, second order effect? ===== 2026-04-01 ===== * motion comparison with undergrad. correlation with param and tools. reho, connectivty/def mode graph? * see Ted S.; fronteirs paper * gyroscop. needs grant * oac /eru; scan data * TODO: send scripts/RA notes on eyetracking (Silk lab; BRIDGE. camera focus) ===== 2026-02-11 ===== * WF * scan_data mounted! * Shim values bad for P3 X; P1 X,Y. DailyQC updates needed for uT/m2 calc * EH * broke apart the main() method in email_latest_flip * created the tests for it as well * next step is to move to cerebro ===== 2026-02-04 ===== * AP downloaded flywheel cli, logged on successfully, reading docs * Singularity image of neuromelanin pipeline seems to be referencing a relative path? * WF: [[::it:qa:dev]] [[https://github.com/NPACore/fw-mrrcqa/compare/efc26bb..main|fw-mrrcqa]] FID automation and weekly upoads ~ 8h - mount update * EH - Started the refactoring process of the main method ===== 2026-01-28 ===== * AP no updates; last week: programmers happy hour - gear for NM on FW. cerebro? not needed ==== EH ==== * cerebro / mrqart * git clone * cron onto cerbro * more tests -- refactor main, no global dict * using template checker; refactor email * Tetra: DA study reproduction ==== WF ==== * SSL resolved; gyrus down; * iFFT rotate kspace w/ CHM code; write to nifti broken ~ 2 h * pscyhopy task for grasping. Alex found EPrime task instead? ~ 3h * [[::it:qa]] matlab plotting: revise for R2019a. fix temperature pull ~ 2h - TODO: file-curate for FID | .5h ===== 2026-01-21 ===== ==== AP ==== * MRSI - Schefer ROIs * NM docker works; bash script tested on oac-moon, testing singularity on CRC (Friday) * Convert to Flywheel gear ===== 2026-01-14 ===== ==== EH ==== ==== AP ==== * MRSI - shaefer ROI, CRLB, * sameria has list of PIs/projects * NM docker works; bash script to run, testing singularity * on /eru * https://fw.mrrc.upmc.edu/ * fw gear: manifest.json (input spec, docker container info); run.py to use flywheel config spec in manifest; CLI * access to ''WPC-8986@cortex'' (''10.48.88.14'') for ''/disk/mace2/scan_data/WPC-8986/'' rsync * ''/etc/host'' edit set ''xnat.mrrc.upmc.edu'' to external ''157.229.38.7'' ==== WF ==== [[:it:flywheel:development#api_key]] - FWHM for fw-mrrcqa [[https://github.com/NPACore/fw-mrrcqa/compare/787a7a0~3..main|diff]] ~ 8h! - [[https://github.com/WillForan/duo-hotp/|duo-htop]] (+openconnect for pitt) new bug ~ 2h [[https://xkcd.com/1205/|worth it?]] - LLM agents thoughts to Alex ~2h(!) * Want case studies, prompt/org suggestions; time tracking - [[qc]] caught NuMoM sequence issue? ~1h - ''[[:npac:tat2|tat2]]'' to [[https://github.com/lncd/dR2star|dR2star]], hand off to UMN ~ 4h ===== 2026-01-07 ===== ==== AP ==== - Built docker container and singularity image - Can clean up and add comments or just be done. - What is the next project / are the next steps? ==== EH ==== ==== WF ==== * matlab -> docker ([[https://github.com/NPACore/fw-mrrcqa/commit/215cd54ac3b90dc839124b7fe3cff8c26dfbb80c|matlab-branch commit]] results = compiler.build.standaloneApplication("../Program/dostat.m"); compiler.package.docker(results, 'ImageName', 'fwmrrcqa-mlbase') system('docker image ls --format=json fwmrrcqa-mlbase|jq -r .Size') % 3.57GB * TODO: cerebro2 access to andrew/ via npac@... ===== 2025-12-17 ===== ==== WF ==== - [[:it:qa]] false positive. need to support multiple templates ? ~ 1h - [[:it:qa:snr]] fast tSNR, re-benchmark. matlab 40s, octave hour+!? work on matlab container ~ 3 h * [[https://github.com/NPACore/fw-mrrcqa/commit/e24089c6e7477062ff25c3d6c421e36dda492c30|diff]] - use mcc's requiredMCRProducts.txt for smaller run time? - [[it:flywheel:infrastructure]] cert [[https://github.com/NPACore/hpc-env/commit/d7368869d28404baee627d7f8787115ebc011700|diff]] curl -Iv https://fw.mrrc.upmc.edu |& grep expire # expire date: Jan 4 23:59:59 2026 GMT /raidzeus/flywheel/certs/2026/setup_ssl.bash -> /raidzues/src/hpc-env/flywheel/setup_ssl.bash ==== EH ==== MRQART daily header-QA email cleanup & signal reduction ~ 2h - cleaned up mismatch formatting (expected → got), incl. FA / TA deltas - cut down email noise while keeping marquee vs any-diff behavior - made red-day emails quicker to scan for protocol issues * [[https://github.com/NPACore/mrrc-hdr-qa/compare/main...static_html|diff]] Automated nightly template rebuild & MIA fixes ~ 3h - rebuild template_by_count at runtime to avoid stale templates - split MIA reporting into onboarded-study vs non-onboarded study - removed false-positive MIAs from lagging template updates Non-conformance aggregation & post-hoc filtering ~ 2h - grouped parameter mismatches with counts across acquisitions - capped per-sequence examples without losing useful signal - filtered out post-hoc series (>200) from compliance checks ----- * report item per ''study/subj/sequence'' instead of ''study/sequence'' * counts like (''number sequences matching template'', ''number match non-conforming parameter'' / ''total number of sequeneName in study'') * move from rhea to npac@cerebro2? cron @ 6am? * pullout white/black list, params-to-check into config file? * start removing from blacklist. want to check EPI and fmaps * tests with examples * multiple templates for single acq? (nav_setter) * don't overwrite existing templates? * eventual interface for templates ==== AP ==== - Built docker container and singularity image - Can clean up and add comments or just be done. - What is the next project / are the next steps? ------ * NM for all sequences at MRRC? cerebro2 or CRC slurm? * NM QC pulled out to work w/o batch (''--participant-label sub-001'' still makes QC pdf?) * MRSI [[https://github.com/LabNeuroCogDevel/QPASA_slice-warp|QPASA]] headless mode * Flywheel QA (zshim) -> cron, integrate tsnr, wiki page update? ===== 2025-12-10 ===== ==== EH ==== - Refined MRQART daily email * removed footer noise, tightened message formatting, swapped to SequenceType logic | 3h - Added full blacklist + filtering rewrite * implemented prefix-based exclusion, verified against database-wide stats | 6h - Rebuilt template_by_count * validated MP2RAGE/ACE templates, resolved all “no template defined” cases | 5h - Next week * Add optional template-diff reporting (highlight changes over time) | 4–5h - maybe more spacing; html * Automate nightly template rebuild + logging into run_mrqart_daily | 3h * Explore improving TA (scan length)/FA mismatch descriptions | 2–3h * Also looking for feedback to make the email less dense - master.xlsx lookup for physicist -- eventually send to them. email per study. - git commit - 2026: talk to phys; shot of image - Brain^wpc-8796 / test / ABCD_T1w_setter (Series 6): (3/5 bad) expected / have iPAT: null / p2; got R: -0.0008 0.4671 -0.0594 -0.8822 / 0.04 -0.02 0.01 ==== WF ==== - dokuwiki [[:it:this_wiki]] * [[https://github.com/dokuwiki/dokuwiki/pull/4490|32bit support for ipv6 into upstream]] | 5h * login button ''ewi:/wiki-mrrc/lib/tpl/readthedokus/tpl_parts'', [[https://github.com/NPACore/dokuwiki-template-readthedokus/blob/main/tpl_parts/tpl_mobileheader.html#L12|tpl_mobileheader.html]] ([[https://github.com/NPACore/dokuwiki-template-readthedokus/commit/3f544f09680af91c569e905d60310bf7ea47eae4|diff]]) | 1h - [[:it:qa]] QA reporting T1w DB corrupt!? | 2h {{.:pasted:20251210-104226.png}} - BIDS fw example | 1h * [[:it:qa]] fast tSNR ~ 6h * [[:it:mrschedule]] MRRC schedule DB fixed slots ~ 6h ==== AP ==== * [[:npac:neuromelanin]] NM_Pipeline built and (almost) running. Current error ''line 31: ./for_redistribution_files_only: Is a directory.'' This is for the ''ENTRYPOINT'' command. Tried with and without ''./'' in ''./for_redistribution_files_only/run_NM_Pipeline.sh'' * TBO Task eye tracking at Bridge. (See email, waiting to hear from Helmet). Probably can try again with arguments to control executable? * [[:npac:icaaroma]] Updates to ICA-AROMA? Can it be done in native space for precision psychiatry? Would this be useful? Hallquist BrainGnomes implements old ICA-AROMA pipeline already. * Deepak's idea: Can MRRC make PDF's of scan parameters compatible with NDA upload spreadsheet format? - https://github.com/WillForan/bids2nda - Office of Data Management Adrian Sonheim (sleep) ===== 2025-12-03 ===== * EH: * condensed the email * filtered using SequenceName -> swapping to SequenceType * TODO: * listing allways flagged, finding SequenceType, blacklist | 5h * template sql code, run night? diff on the templates? | 1h + [10h] * (Bridge) FW job stuck for week+: rerun with logging (fw issue) * github repo for hafman code (xcpd scripts for extra atlases) * AP: * Docker image built, can run, but some path errors remain. Very close! * EyeLink calibration working at Bridge. Grabbed code from Eyelink Testing scripts. Possibly the issue was the wrapper script. "O" in CAVAO sequence wasn't working. Ideas? * find path to control exe, WF TODO linux path * esc, ctrl-c; host pc * monitor: psychopy/pylink demo window(0 vs 1) * Triggy at some point * Differences in old vs. new pipeline for Deepak's TMS study (probably worth discussing) * ENACT (WPC-8986) ME fMRI AXCP (task) + resting; * See UNC IT slack; brainomes * fmriprep 25 Vs xcpd 0.5 vs 0.11; dlPFC cor. .3-.6 thres -- was .9;; T1w wrong space for TMS software; use warren's for last subject * ICA aroma * validating * vs fmriprep cosine/regessors (CompCor regressors) * WF * SPM DWI BIDS [[https://github.com/spm/spm/pull/87/files|PR#87]] accepted * tSNR mask working. 10x? speedup * [[https://github.com/NPACore/fw-mrrcqa/compare/dcd5b9e7db8fd83300bcd43410e31c493d84e35e...main|diff]] * [[:it:fire]] * working Prisma1<->ptx at scan time https://github.com/NPACore/fire-recontwix * PR for tests in python ismrmrd-server [[https://github.com/kspaceKelvin/python-ismrmrd-server/pull/14|PR#14]] * TODO: m tilt and MRSI slice position injector * Cerebro2 as NIST800-171 environment okayed by MR council * Adding git diff * ''git log -p'' * {{.:pasted:20251203-122538.png?300}} * protected main branch? PR workflow? ===== 2025-11-19 ===== * EH: * email not sending. test with python oneline * condense * oneday, all projects * ABCD T1 vnav -> two filtered (dervied) raw data. * AP: * Docker image is building, need to add .dockerignore file and/or move base docker file and subdirectories up a level to save space in the build (~25GB) * Deepak had a question about the MRSI Figure 1 ROI –> MNI Space. There was an overlap of the LNCD data and Sarpal data that got misplaced when Warren left. Deepak understood Warren to be working on this with Will. * WF TODO: 3dMean of MRSI ROI for heatmap (need paths to volume) * Sarpal 7T preproc /eru/shares/sarpal/test/7T/bids_correct/output/sub-2326/ses-20181019/rest/rest.nii.gz skullstrip only applied to first volume, all other volumes are empty! rest.nii.gz corrupt? can use rest.3/rest.nii.gz to restore * hallquist can run on moon for new/missing subjects. see logs of existing for command line inputs * Eye tracking testing at Bridge Thursday at 4p? * WF TODO: linux path to control SDK – sample experiments * Triggy at some point ===== 2025-11-12 ===== * QA emails sent! switch to all projects, just yesterday. alt all okay email * eventually link to project template * EyeTracking BRIDGE still an issue * see C:\Program Files (x86)\SR Research\EyeLink\SampleExperiments\ from the EyeLink SDK * Dockerfile; docker build $PWD for include files only * Deepak: MRSI 7T; spack setup w/module load; MP2RAGE UNICORT ===== 2025-10-29 ===== * alerts - nice looking html interface! emailing for next week * NM pipeline - containerize, add masks * flywheel was down at MRRC "upgrade may have been stuck in pending" ===== 2025-10-22 ===== * mrrc-hdr-qa/mrqart/email_latest_flip.py * loop over all new: acq2sqlite.find_acquisitions_since * NM pipeline - matlab container, bash. Phillips ''billy jewel''? helmet * FW (CRC into RHEL 9), clpipe deprecating, braignomes in dev? * Feb-April new 7T * process, async standup ===== 2025-10-15 ===== * eye tracking - didn't work; cal crash. 2nd monitor off. * watch out: axis flip from scanner if FOV angle changed to extreme * review mrqart - loop over scans_since, template_check, report conforming ===== 2025-10-09 ===== * P2 body coil replaced; new zshim? rerun stats next week * can we pull (all EPI sequence z-shims) from DB? * how variable? more variable over any time? * [[:it:mrqart]] | * prev: script / days - move to python package! ''uv init .'' (name mrqart instead of *hdr*) | 7 - use ''DB = acq2sqlite.DBQuery()'' - docstrings of template_checker and acq2... if/when needed - check with ''make .test.pytest'' - pyproject.toml console-script => ~/.local/bin (install ''mrqart'') ===== 2025-10-01 ===== * [[:it:mrqart]] | prev5? * email setup! sending via ''sendmail'' -- add andrew to list - in cron on rhea * cron to mrqart * which fields? * mass pull: **Flip Angle** for just **bold** - (standard, median->mode value) TR, TE, FOV, ...; **Protocol Name**: t2pe * does email send to list? * email format: date, scan, what mismatches * loop over projects - filter by date * eventaull into database * RT motion plots, thresholds (andrew?) -- send to code * [[:npac:neuromelanin]] | prev1-5? * MacOS vs Linux Runtime? Including dependent files * [Seq meeting this afternoon] * reproin/BIDS push to FW pilot users ideas? - preprocess alex outstanding dataset. follow up (may be BIDS) * P2 Rx failure? Tx box unstable? [need to check NM data] * Flywheel debrief! ===== 2025-09-24 ===== * [[:data:qc:interactive]] - EH date subset; added scroll and toggle. ([[https://github.com/NPACore/fw-mrrcqa/commit/64eb820d7eaf1ac8cc9e635398cc44c91d9eb094|commit]]) * [[:it:mrqart]] cron db not running since cerebro->warewulf! * check on email code. test code / cli call that flags an issue | 5 * [[:npac:qc:zshim]] * DONE: email update. Makefile change problematic (2-3) * DONE: recalc thresh; 1-month 1sd of zshim (from csv P2) | 2+1 * add to repo, docs, maybe setup cron | 1 * cron: read dataframe from https: url via pandas * [[:it:mrqart]] summarize different parameters inside seq (flip angle). identify strange values. start with bold. * send an email example report | 6 * [[:npac:neuromelanin]] * docker w/matlab runtime like r2prime | 2h; 1-5 * matlab and docker (cf. podman) setup -- use ''nucleus'' * example [[https://github.com/LabNeuroCogDevel/r2prime-prisma/blob/main/Dockerfile|r2prime-prisma/Dockerfile]] ===== 2025-09-17 ===== * 96G 4min tar over globus (OAC server to clevland) * singularity + xcpd -> * file curator on fmriprep output trigger email. - email server connection a problem from google cloud. - cf. cron ''fw ls'' check and mail * [[:npac:qc:zshim]] * email update | 1 * DONE: interactive plot "parse DATE as date" * recalc thresh; 1-month 1sd of zshim (from csv P2) | 2+1 * [[:npac:neuromelanin]] * docker w/matlab runtime like r2prime | 1-5 * example data into repo? bad idea w/o skull strip? ''git-lfs'' -- use mni warped to subject * connect a la [[:data:crc]] - https://windows.cloud.microsoft/ -> pitt sso -> Psych AVD -> FastX * FW:Mike Gross - NDA vs BIDS * [[:data:flywheel:gears]] annotate/document flywheel, ''complete_test''. From workshop: file rename instead of bids-curate * [[:it:mrqart]] summarize different parameters inside seq (flip angle). identify strange values. start with bold. * send an email example report | 6 ===== 2025-09-10 ===== 👋 Hi Dr. Moon & welcome Andrew! * flywheel workshop: push for cloud only. ITK-snap app with launcher. * tools/infrastucture * move to dokuwiki (?) from confluence (assuming EWI) [[https://tetra-argo.atlassian.net/wiki/spaces/M/pages/56819713/Weekly|tetra confluence]] * issue tracker / "story points"; docs-in-code (sphinix) integration * email config pytest updates (EH) * deid [[https://github.com/pydicom/deid/pull/285|PR pending]] (WF);a Michelle's question still open (email) * [[https://services.pitt.edu/TDClient/33/Portal/Requests/TicketRequests/TicketDet?TicketID=9Q-rgirmiZY_|EWI pending]] * NCANDA pinged about [[::mr:task:trigger]] * NPAC: prioritize. - [[:npac:qc]] * [[:npac:qc:zshim]] * 🌟 email robustness: send to MR accounts (EH: works w/ ''pytest tests/'') * 🌟 adjust P2 threshold after TX box replacement? * 🌟 interactive plot [[::data:qc:interactive]] * rewrite with linux-inotify instead of flywheel * [[:npac:qc:tsnr]] -- rewrite for speed * matlab circshift (wrap around) vs ANTs alignment (cut off FOV) * [[:npac:qc:FID]] channels (kspace signal) * [[:npac:qc:MRIQC]] - BIDS * 🌟 reproin push * 🌟 annotate/document flywheel, ''complete_test''. From workshop: file rename instead of bids-curate * lncdtools bids gear - dockerize lncdtools (in progress) * flywheel gear for bids tools * flywheel gear for tat2 - NM (AP/Deepak) * 🌟 dockerize + flywheel-ize * kernel density out of matlab (into AFNI 3dROIstat). also useful for tsnr * rewrite in shell/python * Jones: value into sheet, reverse, code to shift image (ADEPT study) - deid.py * 🌟 demo/docs - bids2nda - document, push to users - MRQART * add motion * protocol QA in cron/DB build * push groups to use? * different users using different params: BOLD - 🌟 summarize different parameters inside seq (flip angle). identify strange values. start with bold. - Yarra @ NYU, overnight worker - TWIX transfer and processing. kspace recon. future proofing . offline high-load computation. * Yarra on "ptx" computer; high accel MPRAGE * OpenRecon -- "inside" scanner * FIRE Siemens WIP - recon + fmriprep. docker. run external or internal - Cerebro2 HPC maintaince: update from debian12 to 13 (bookworm->trixie) - wiki conviences * added collapse button * keybindings to prose editor * php-scripts for pulling out log (could be plugins): [[:it:log]] * ☁️integrate with git-bug * ☁️integrate with doc generators - ☁️ doxygen + sphinix: static site generator for scripts comments; monorepo - ☁️ ML + MR sequence development * Deep Learning for reconstruction (kspace). better noise, faster sequence * need math background; vs. MR signal model for reconstruction. snapshot vs cumulative time. * eigen vector fit to remove noise: data + model to clean * optimization over large datasize (slow) vs. train and apply to inc speed (order of mag) * increase resolution * optimize code for GPU/CPU resources. CUDA/tensor/pytourch vs matlab optimizer * ML pulse sequence programming. RF, gradient, acq signal encoding (amplitude, phase) * EPI: train on echo time, matrix size, ... parameters - pitt email to andrew for slack; === EH === * delete workign directroy free lots of data * xcpd - shen atlas (empty files), rerun (workdir) populates tsv. workdir=$(mktemp -d /tmp/XXXX) * https://github.com/PennLINC/xcp_d * tar search for dicoms * https://github.com/ajeetdsouza/zoxide z-jump to dirctory on partial match of name * shim_notify_test.py -- test, then upload for all * interactive plot [[::data:qc:interactive]] -- 2 months * autodocs for file-curate * Enigma upload dicoms, waiting for BEAM dcm (maybe on flywheel)