Prerequisites

Expected and helpful background knowledge for joining an event on this topic.

Background

  • Expected: Familiarity with Git and collaborative repository platforms
  • Expected: Comfort writing small programs
  • Helpful: Basic understanding of reproducibility concepts

Scientific analyses often involve repeating operations across files, choosing steps based on metadata, and running calculations in a particular order. As a project grows, expressing repetition, decisions, and dependencies explicitly helps an analysis remain understandable and repeatable.

Why This Matters

A research workflow contains knowledge that can easily remain implicit: which files belong together, which parameters were used, which steps must run first, and how a result was produced. Making these decisions explicit reduces reliance on memory and manual coordination, helping both you and your collaborators understand and repeat the analysis.

Automation makes checks especially valuable because the same operation may run across many datasets. Defined expectations, reviewed reference outputs, and recorded provenance help you notice unexpected changes and investigate their causes. Together, these practices make workflows easier to inspect, update, and reuse.

Python Approach

For this approach, familiarity with Python and Jupyter notebooks is required. Consult the event page for its specific setup and requirements.

Using practical research examples, you will process files with loops and conditionals, extract metadata from filenames, and turn analysis steps into configurable commands. You will organize environments and tasks with Pixi, connect steps visually with Xircuits, and coordinate scripts and notebooks using Snakemake and other automation tools. DataLad will help connect results to versioned data and recorded commands, while acceptance and approval testing provide ways to check workflow behavior and outputs.

Questions We Can Explore Together

Events

Explore upcoming dates and details from previous events.

–

Reproducible and Portable Scientific Workflows

Starts at 4 sessions

This four-week online course shows how to turn scientific analyses into explicit, reproducible workflows. Participants will separate code from configuration, track data and …

Technology Radar

Our view of the tools and practices relevant to this topic. Recommendation and teaching availability are separate.

Provisional positions are editorial proposals awaiting group review.

Recommendation rings

  1. Recommend
    A recommended choice for relevant research work.
  2. Explore
    Promising or used internally; suitability or teaching integration is still developing.
  3. Assess
    Worth investigating before trying or recommending.

Kinds of item

  • Libraries
  • Languages & frameworks
  • Tools & platforms
  • Practices
  • Data formats

Items in this view

Recommend

Git

Distributed version control system for tracking changes in source code

Recommend Provisional

Tools & platforms · Teaching

5 topics with recorded teaching · 20 recorded and scheduled offerings

Our View

Git is the industry standard for version control, enabling researchers to track changes, collaborate effectively, and maintain a complete history of their work. It's essential for reproducible research and collaboration.

Recorded in topic offerings; consult the dated event links for availability.

19 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: AI-Assisted Software DevelopmentArrays and Numerical ComputingAutomating Research WorkflowsDeveloper EnvironmentRobust and Reusable SoftwareWebsite Development

JSON

Lightweight text-based format for storing and exchanging structured data with nested hierarchies

Recommend Provisional

Data formats · Not currently taught

0 topics with recorded teaching · 0 recorded and scheduled offerings

Our View

JSON is the standard for structured data exchange on the web and in APIs. Its human-readable format with support for nested structures makes it perfect for configuration files, metadata, and hierarchical data that doesn't fit in CSV.

Relevant to the topic landscape; no explicit recorded offering is currently attributed to this item.

0 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Programming and Data AcquisitionAutomating Research WorkflowsDataframes and Statistics

Interactive computing environment combining code, visualizations, and narrative text

Recommend Provisional

Tools & platforms · Teaching

6 topics with recorded teaching · 17 recorded and scheduled offerings

Our View

Jupyter notebooks allow you to combine code, results, and explanations in a single document. This makes them perfect for exploratory data analysis, teaching, and creating reproducible research workflows. You can iterate quickly, visualize results inline, and document your thought process as you work.

Recorded in topic offerings; consult the dated event links for availability.

15 past offerings · 2 upcoming offerings

Related themes: Scientific Computing

Related topics: Arrays and Numerical ComputingAutomating Research WorkflowsDataframes and StatisticsDeveloper EnvironmentMachine LearningRobust and Reusable SoftwareWebsite Development

Pandas

Powerful data analysis and manipulation library for Python

Recommend Provisional

Libraries · Teaching

5 topics with recorded teaching · 16 recorded and scheduled offerings

Our View

Pandas provides intuitive data structures (DataFrames) for working with tabular data, making it easy to clean, analyze, and visualize datasets. It's the go-to tool for exploratory data analysis and handling experimental results in research.

Recorded in topic offerings; consult the dated event links for availability.

14 past offerings · 2 upcoming offerings

Related themes: Neural Data AnalysisScientific Computing

Related topics: AI-Assisted Software DevelopmentArrays and Numerical ComputingAutomating Research WorkflowsDataframes and StatisticsLocal Field Potential (LFP) AnalysisNeural Spike Analysis

pathlib

Python standard-library module for working with filesystem paths as objects.

Recommend Provisional

Libraries · Not currently taught

0 topics with recorded teaching · 0 recorded and scheduled offerings

Our View

pathlib makes path handling more readable and portable than manually joining path strings.

Relevant to the topic landscape; no explicit recorded offering is currently attributed to this item.

0 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Arrays and Numerical ComputingAutomating Research Workflows

Pixi

Modern, fast package manager built on conda-forge

Recommend Provisional

Tools & platforms · Teaching

2 topics with recorded teaching · 10 recorded and scheduled offerings

Our View

The repository already uses Pixi to manage its toolchain. Project dependency declarations, lock files, and tasks make it a practical candidate for reproducible research environments.

Recorded in topic offerings; consult the dated event links for availability.

9 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsDeveloper Environment

pytest

Python testing framework for writing simple and scalable test cases

Recommend Provisional

Libraries · Teaching

3 topics with recorded teaching · 5 recorded and scheduled offerings

Our View

pytest makes it easy to write tests for research code, catching errors before they affect results. Its simple syntax and powerful features help ensure reproducibility and build confidence when refactoring analysis pipelines.

Recorded in topic offerings; consult the dated event links for availability.

3 past offerings · 2 upcoming offerings

Related themes: Scientific Computing

Related topics: AI-Assisted Software DevelopmentAutomating Research WorkflowsDeveloper EnvironmentRobust and Reusable Software

Python

General-purpose programming language widely used for scientific computing and research software.

Recommend Provisional

Languages & frameworks · Teaching

10 topics with recorded teaching · 33 recorded and scheduled offerings

Our View

Python combines a readable language with a mature scientific ecosystem, making it a practical default for research data analysis, automation, and teaching.

Recorded in topic offerings; consult the dated event links for availability.

26 past offerings · 7 upcoming offerings

Related themes: Neural Data AnalysisScientific Computing

Related topics: Programming and Data AcquisitionAI-Assisted Software DevelopmentArrays and Numerical ComputingAutomating Research WorkflowsCalcium Imaging AnalysisDataframes and StatisticsDeveloper EnvironmentLocal Field Potential (LFP) AnalysisMachine Learning(M)EEG AnalysisNeural Spike AnalysisRobust and Reusable Software

Record and reproduce the software environment used for an analysis or application.

Recommend Provisional

Practices · Teaching

0 topics with recorded teaching · 0 recorded and scheduled offerings

Our View

Declared dependencies and lock files help collaborators rerun analyses and distinguish environment changes from changes in scientific results.

Covered through our computing environment and research workflow topics; individual offerings are not yet attributed to this practice.

0 past offerings · 0 upcoming offerings

Related themes: Neural Data AnalysisScientific Computing

Related topics: AI-Assisted Software DevelopmentArrays and Numerical ComputingAutomating Research WorkflowsCalcium Imaging AnalysisDataframes and StatisticsDeveloper EnvironmentLocal Field Potential (LFP) AnalysisMachine LearningRobust and Reusable Software

Explore

Apptainer

Container runtime for packaging Linux research software in portable, reproducible environments.

Explore Provisional

Tools & platforms · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

Apptainer makes it practical to move complex scientific software between laptops, workstations, and high-performance computing systems without asking researchers to rebuild every dependency by hand.

Recorded in topic offerings; consult the dated event links for availability.

0 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research Workflows

Package and environment manager for Python and scientific software

Explore Provisional

Tools & platforms · Teaching

2 topics with recorded teaching · 11 recorded and scheduled offerings

Our View

Useful for existing conda-based workflows and scientific package compatibility. Compare it with Pixi when designing new reproducible project environments; the group should review which approach to teach as its default.

Recorded in topic offerings; consult the dated event links for availability.

11 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsDeveloper Environment

DataLad

Tool for versioning, publishing, and sharing research datasets and analysis workflows.

Explore Provisional

Tools & platforms · Teaching

2 topics with recorded teaching · 2 recorded and scheduled offerings

Our View

DataLad helps research groups keep data, code, and provenance connected while collaborating across machines and institutions.

Recorded in topic offerings; consult the dated event links for availability.

1 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsRobust and Reusable Software

DVC

Data and model versioning tool for machine learning and reproducible data pipelines.

Explore Provisional

Tools & platforms · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

DVC gives Git-like workflows for files that are too large or too changeable to store directly in Git repositories.

Recorded in topic offerings; consult the dated event links for availability.

1 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsDeveloper Environment

fsspec

Unified filesystem interface for Python providing consistent API across local and remote storage

Explore Provisional

Libraries · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

fsspec provides a single, consistent interface for working with files whether they're on your local disk, in the cloud, or on GitHub. This means you can write code once and access data from multiple sources without changing your file operations.

Recorded in topic offerings; consult the dated event links for availability.

0 past offerings · 1 upcoming offerings

Related themes: Neural Data AnalysisScientific Computing

Related topics: Automating Research WorkflowsCalcium Imaging AnalysisDataframes and Statistics

Version control platform for research data

Explore Provisional

Tools & platforms · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

GIN combines Git-like version control with storage optimized for large research datasets. It's specifically designed for neuroscience and other data-intensive fields where you need both version control and efficient handling of large files.

Recorded in topic offerings; consult the dated event links for availability.

1 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsRobust and Reusable Software

GitHub

Web-based platform for hosting and collaborating on Git repositories

Explore Provisional

Tools & platforms · Teaching

5 topics with recorded teaching · 18 recorded and scheduled offerings

Our View

GitHub is the de facto standard for sharing and collaborating on code. It provides excellent tools for code review, issue tracking, and project management. Most scientific software is hosted on GitHub, making it essential for reproducible research.

Recorded in topic offerings; consult the dated event links for availability.

18 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: AI-Assisted Software DevelopmentArrays and Numerical ComputingAutomating Research WorkflowsDeveloper EnvironmentRobust and Reusable SoftwareWebsite Development

Automation platform for running tests, checks, and deployment workflows from GitHub repositories.

Explore Provisional

Tools & platforms · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

GitHub Actions lets research teams automate repetitive checks so shared code remains easier to trust and maintain.

Recorded in topic offerings; consult the dated event links for availability.

1 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsRobust and Reusable Software

GitLab

Git-based collaboration platform for repositories, issues, reviews, project boards, and automation.

Explore Provisional

Tools & platforms · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

GitLab combines code hosting, project coordination, and CI/CD in one place, making it useful for research groups that need transparent collaboration around shared software.

Recorded in topic offerings; consult the dated event links for availability.

0 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsDeveloper EnvironmentRobust and Reusable SoftwareWebsite Development

GitLab CI/CD

GitLab's built-in automation system for running tests, checks, builds, and deployments from repository changes.

Explore Provisional

Tools & platforms · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

GitLab CI/CD turns project expectations into repeatable checks, helping research teams catch broken tests, missing dependencies, and packaging problems before changes are merged.

Recorded in topic offerings; consult the dated event links for availability.

0 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsRobust and Reusable Software

Python tool for measuring memory usage line by line or over time.

Explore Provisional

Libraries · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

memory-profiler helps researchers see where an analysis allocates memory, which is often the first step toward making large-data workflows reliable.

Recorded in topic offerings; consult the dated event links for availability.

1 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Arrays and Numerical ComputingAutomating Research WorkflowsRobust and Reusable Software

Platform for open and reproducible research project management

Explore Provisional

Tools & platforms · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

OSF provides a comprehensive platform for managing research projects openly. It supports preregistration, data sharing, and connects with many other tools (GitHub, figshare, Dropbox). It's designed specifically for research workflows and promotes transparency.

Recorded in topic offerings; consult the dated event links for availability.

1 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsRobust and Reusable Software

ownCloud

Open-source file sync and share platform, used via Sciebo at German universities

Explore Provisional

Tools & platforms · Not currently taught

0 topics with recorded teaching · 0 recorded and scheduled offerings

Our View

OwnCloud (via Sciebo) provides secure cloud storage that's hosted by German universities, ensuring GDPR compliance and data sovereignty. The Python client makes it easy to automate dataset downloads and share data with collaborators.

Relevant to the topic landscape; no explicit recorded offering is currently attributed to this item.

0 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research Workflows

Papermill

Tool for parameterizing and executing Jupyter notebooks programmatically

Explore Provisional

Tools & platforms · Teaching

2 topics with recorded teaching · 3 recorded and scheduled offerings

Our View

Papermill lets you run the same analysis notebook with different parameters, making it easy to process multiple subjects, conditions, or datasets. It's perfect for batch processing while keeping your analysis in the familiar Jupyter format.

Recorded in topic offerings; consult the dated event links for availability.

2 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsDeveloper Environment

Pydoit

Python task automation tool for defining and executing task dependencies

Explore Provisional

Tools & platforms · Teaching

1 topics with recorded teaching · 2 recorded and scheduled offerings

Our View

Pydoit brings make-style build automation to Python with a simple, Pythonic interface. It's perfect for automating repetitive analysis tasks, managing data processing pipelines, and ensuring your analysis steps run in the correct order.

Recorded in topic offerings; consult the dated event links for availability.

1 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research Workflows

Sciebo

Secure cloud storage for research data in German universities

Explore Provisional

Tools & platforms · Teaching

2 topics with recorded teaching · 10 recorded and scheduled offerings

Our View

Sciebo provides GDPR-compliant cloud storage hosted in Germany, specifically designed for academic use. It offers generous storage quotas and integrates well with university authentication systems, making it ideal for storing and sharing research data.

Recorded in topic offerings; consult the dated event links for availability.

10 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Arrays and Numerical ComputingAutomating Research WorkflowsDeveloper Environment

Snakemake

Python-based workflow management system for creating reproducible and scalable data analyses

Explore Provisional

Tools & platforms · Teaching

1 topics with recorded teaching · 3 recorded and scheduled offerings

Our View

Snakemake helps you build reproducible analysis pipelines that track dependencies automatically. It ensures your analyses run in the correct order, reruns only what's necessary when data changes, and scales from your laptop to compute clusters.

Recorded in topic offerings; consult the dated event links for availability.

2 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research Workflows

timeit

Python standard-library module for measuring execution time of small code snippets.

Explore Provisional

Libraries · Teaching

1 topics with recorded teaching · 2 recorded and scheduled offerings

Our View

timeit gives researchers a simple, repeatable way to compare small implementation choices before investing in heavier profiling or optimization work.

Recorded in topic offerings; consult the dated event links for availability.

1 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: Arrays and Numerical ComputingAutomating Research WorkflowsRobust and Reusable Software

Microsoft-supported way to run Linux command-line environments directly on Windows.

Explore Provisional

Tools & platforms · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

WSL helps Windows users participate in Linux-oriented research workflows without needing a separate machine, making command-line tools, package managers, and pipeline software easier to use.

Recorded in topic offerings; consult the dated event links for availability.

0 past offerings · 1 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsDeveloper Environment

Assess

hvPlot

High-level plotting API providing interactive visualizations for pandas, xarray, and other data structures

Assess Provisional

Libraries · Teaching

1 topics with recorded teaching · 1 recorded and scheduled offerings

Our View

Assess whether this fits the group’s research and teaching workflows. hvPlot makes creating interactive visualizations as easy as using pandas' built-in plotting, but with interactivity out of the box. It's perfect for exploratory data analysis, allowing you to zoom, pan, and hover over data points without writing complex plotting code.

Recorded in topic offerings; consult the dated event links for availability.

1 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research Workflows

nbdev

Framework for building tested, documented Python packages from Jupyter notebooks.

Assess Provisional

Languages & frameworks · Teaching

0 topics with recorded teaching · 0 recorded and scheduled offerings

Our View

Assess whether this fits the group’s research and teaching workflows. nbdev can help notebook-heavy projects become importable, documented, and tested packages while preserving an exploratory authoring workflow.

Recorded in topic offerings; consult the dated event links for availability.

0 past offerings · 0 upcoming offerings

Related themes: Scientific Computing

Related topics: Automating Research WorkflowsRobust and Reusable Software

Let's get in touch

For tutoring, collaboration, or follow-up support, contact the team. We are happy to help you connect workshop ideas with your own research practice.

Privacy information for email contact