🏗️ Development infrastructure team

• Worked together to set up development environments for the first time

• Explored using devcontainers to reduce startup burden for new contributors

  This is an interoperable development container that automates the dev environment setup for JupyterGIS development. It works both on Hub infrastructure and on VSCode / Codespaces without any additional tinkering! This means that users, regardless of their host OS, can build and iterate on JupyterGIS either locally (using Docker or Docker + VSCode) or in the cloud (on managed Hub infrastructure or GitHub codespaces). A byproduct of this work was the development of a Dev Container Feature that provides a way for Jupyter-based images to play nice with devcontainer-based tools without fuss, something I have been annoyed with for some time.

• Discussed the potential of using a JupyterHub for developing on JupyterLab extensions

📚 Documentation team

Our hackathon topic brought together a diverse set of backgrounds and personas of potential JupyterGIS users to test out and improve existing documentation and tutorials. Several pull requests, bug reports and feature suggestions were created and the JupyterGIS project gained another 9 testers. Repeating this experiment by giving this task to different types of user working groups is one great way to help build a roadmap and envision the future of this project.

• Tested JupyterGIS and reported bugs (#869)
• Suggested new features (#871, #876)
• Updated the structure of the JupyterGIS examples with a “guided tour” feel (#872, #878 #875)
• Tested JupyterGIS tutorials for accuracy and clarity, and submitted fixes (#873)
• Reviewed documentation prose and inline documentation (like docstrings) for clarity and errors, and submitted fixes! (#874, #867, #868, #873, #864)

🐍 Python API ergonomics team

We worked on getting data out of the map layer, manipulating the data and making updates to the map. We showed this was possible through two different workflows: toggling layer visibility and getting GeoJSON from the layer source and turning it into a geopandas object.

Please see #877 for this team’s prototype!

🗺️ Strategy team

We followed a detailed user story focused on biodiversity and identified some gaps and opportunities. We thought about what problems JupyterGIS should and should not try to solve and identified overlap with things Notebooks are good at and many researchers are comfortable doing in Notebooks – analysis. By focusing more on visualization, an area where many practitioners already struggle and a reason they open QGIS, we feel we can more quickly deliver a product that can be used in daily work. We also identified a robust STAC search implementation and GUI as something alternatives do not excel at, and a possible “killer feature”.

• What should JupyterGIS be? What should JupyterGIS not be?
  • Should focus on the areas where users need the most help. For example, visual exploration is the most broadly-reported challenge.
  • Should not focus on drawing users away from Jupyter Notebook analysis workflows that are working for them and are well-served by Notebooks. For example, providing utilities in the visual environment that serve Notebook workflows: drawing or calculating areas of interest and moving those back over to the Notebook, synchronizing changes in Python data objects over to the map. Not trying to reimplement the QGIS/ArcGIS model builders unless we have a strong value-add.

  

• Why do people leave QGIS?
  • Scaling analyses or performing analysis on big datasets
  • Repeatability & reproducibility with a Notebook or script
• Why do people come back to QGIS?
  • Exploring results
  • Making a map to share or tell a story

❤️ Thank you 🙇

Thank you so much to our wonderful hackathon participants for bringing a collaborative energy and creating a supportive atmosphere that made it fun and safe to take on hard problems!

Infinite thanks to our fantastic team of facilitators from the Berkeley Institute for Data Science (BIDS) and the Schmidt Center for Data Science and Environment (DSE) who curated equipment and a comfortable environment, solved problems, provided delicious caffeine and meals, expertly facilitated discussions, and more.

Finally, thanks to the Berkeley Institute for Data Science (BIDS) for hosting us at their wonderful AI Futures Lab space on day 1! What a view 🤩 🌉

📣 Call to join the community

We want to work with 🫵 you! With your collaboration, we can build less frustrating and more joyful and rewarding ways of working with geospatial data. Here’s how:

📖 Share a story about a workflow that needs improvement!

📆 Join a hackathon or community meeting!

🗨️ Chat with us on Zulip!

🧪 Try JupyterGIS! Where does it (not) meet your needs?

😎 Share your rad vibes and leadership!

💌 Get in touch with our community manager: Zulip, GitHub, email

This post co-created by hackathon participants: Arjun Verma, Brianna Pagan, Brookie Guzder-Williams, Ciera Martinez, Fernando Pérez, James Colliander, Jason Grout, Jon Atkins, Kevin Koy, Kirstie Whitaker, Kristin Davis, Lucia Layritz, Maryam Hosseini, Maryam Vareth, Matt Fisher, Maxwell Taniguchi-King, Maya Weltman-Fahs, Maya Zomer, Michele Tobias, Min RK, Nick Gondek, Qiusheng Wu, Shane Grigsby, Stace Maples, Tyler Marino

🤷 How do I do 💃 The GIS Bounce 🕺 ?

There are many variations on this general workflow. One example we heard several times looked something like this:

1. Open Jupyter Notebook in JupyterHub cloud computing environment for analysis
2. Open QGIS on local machine for validation
3. Set up QGIS project for validation:
   1. Create a new project
   2. Add a basemap
   3. Set projection
4. Write some analysis code in the Jupyter Notebook
5. 💃 Bounce 🕺 to QGIS for validation
   1. Write some code to save analysis output to disk in an interoperable geospatial file format
   2. Download the geospatial data file to the local machine
   3. Add the geospatial data file to QGIS as a new layer
   4. Use the Identify Tool and/or Attribute Table to validate the data
6. 💃 Bounce 🕺 back to JupyterLab to repeat step 4-5 until analysis is complete

🤓 Why not “just” use <insert software package here>?

There are many distinct tasks involved in data validation. For example:

• Confirming that the data is correctly geolocated
• Spot-checking attributes of specific features based on their shape and location
• Spot-checking a value of raster grid cells corresponding with landmarks

Keeping up with programming-language-specific ecosystems of tools and features needed to accomplish these objectives is often not aligned with the goals of the practitioner. Using tools like QGIS helps researchers do research, helps educators educate, without unnecessary complexity getting in their way.

✨ Can we do better without leaving JupyterLab?

We’re trying! 💪

We are currently building a workflow to reduce steps 2, 3, and 5 to one line of code (not including imports 😁). For example, to view an analysis output in a GeoDataFrame:

from jupytergis import show_in_jgis

show_in_jgis(my_geodataframe)

After writing this line of code, JupyterGIS would open in a new JupyterLab window with a basemap and the analysis output loaded as layers. JupyterGIS would then allow users to verify geolocation and validate feature attributes and grid cell values, all under one roof. After validating, users would have the option to save their JupyterGIS project for future reuse!

✋ I want to help!

That’s amazing, you’re the greatest!

The GeoJupyter community is committed to building free and open source software that’s owned by its users. That’s you! 🫵 You can participate by signing up for an interview to share your expertise, opening GitHub issues to request features or report problems, chatting with us in Zulip, working directly with other community members in a virtual hackathon, and/or submitting code changes as GitHub Pull Requests.

We are looking forward to seeing you! 😄

📆 When?

Our first hackathon will be on Wednesday, February 5th at 7AM Pacific Time | 8AM Mountain Time | 4PM Central European Time (see the time in your timezone). Duration 2 hours, and all are welcome to drop in or out at any time.

Hackathons will occur every two weeks!

🫵 Who should join?

You! We are looking for geospatial practitioners and software developers of all kinds to join us to explore possible futures for geospatial data work.

🤝 How to join?

Please join our Zoom meeting from our community calendar. Select the event on Feb 5th to view the details. Beneath the calendar, select “Add to Google Calendar”.

Alternatively, click here to subscribe to the GeoJupyter calendar now!

💭 What you can expect

You can expect to meet brilliant and friendly peers, generate ideas, give feedback, learn awesome stuff, write code, and make friends!

🙇 Thank you!

We hope to see you and learn from you soon!

🔭 Vision

GeoJupyter will consist of a new generation of tools and enhancements to existing libraries supporting exploration of data about our planet. This includes tasks traditionally done with desktop GIS applications, but also analysis pipelines that go beyond typical GIS uses; supporting any dataset about our planet that has spatio-temporal structure, for example climate models or subsurface data acquired with remote sensing instruments.

📝 Process

To achieve our goals, we will:

• Gather information about geospatial data practitioners’ workflows and points of pain or joy.
• Establish new GeoJupyter community spaces and build new partnerships with Earth science and open source communities.
• Develop sacrificial concepts and prototypes to fit our work to community needs.
• Deliver high-quality software solutions, educational materials, and training sessions.

⌚ Where is GeoJupyter now?

We have established a community chat space on Zulip and a community calendar. We are currently scheduling our first series of community meetings, with many more to come.

Sponsored by the European Space Agency, QuantStack is leading development of JupyterGIS (GitHub), a multi-player GIS environment for JupyterLab. We’re imagining new workflows, and we need your creativity, knowledge, and skills!

We have completed over a dozen interviews targeting geospatial data practitioners from diverse backgrounds including education, data user support, Earth science, remote sensing, software engineering, and product development. Some themes we’ve preliminarily identified:

We will follow up soon with a more comprehensive blog post about community insights.

For early development, we have narrowed our focus on the educational user: teachers and students. We will build partnerships with research education communities and university classrooms as early users who will shape GeoJupyter.

GeoJupyter contributors currently include QuantStack, 2i2c, Development Seed, the Schmidt Center for Data Science and Environment, members of the UC Berkeley and other university communities, members of the NASA Earth Science community, and members of the Jupyter community. Thank you all for your amazing contributions! 🙇

🫵 How you can get involved

Participants

• Carl Boettiger
• Fernando Pérez
• Ciera Martinez
• Kevin Koy
• Matt Fisher

Values

We brainstormed our values - not in priority order.

Social / community considerations

• Recent challenge: How to invite new people into our Slack?
  • If we want to postpone a decision, we can just pay for those extra folks.
  • Problem remains. We need a long-term solution for the community.
• Some communities are saying “Screw it, we’ll use Discord”
  • It’s proprietary!
  • Conda-Forge and Apache Arrow on Zulip now.
• Short-term plan?
  • Create #dse-geojupyter on DSE Slack for DSE employees working on GeoJupyter
    • The #jupyter-gis channel can be more independent and free to address its needs
  • We can adopt something in the short-term without worrying too much.
    • Once we don’t fit there anymore, because we’ve generated enough interest, we need to figure out how to scale communication!
    • The whole Jupyter project has a non-ideal communication setup. That’s not going to be solved in two weeks.
    • Being adopted by the Linux Foundation. Fernando stepping down from exec. council.
    • Maybe next year, Jupyter is ready to choose and we jump on whatever they have switched to.
• How do we engage with the open source geospatial community?
  • Large community.
  • Be mindful, careful, of how we enter/interact with that community (we’re “parachuting” in with new ideas)
  • We want to make people happy and excited to work with us!
  • Identify our first users and serve them. Interview our community, with GIF’s help.
  • Framing: “We had a problem in our community, and we came up with this solution.”

Getting started ideas

• Set up interviews!
• Work with Geospatial Innovation Facility to develop use cases to guide the development of the tool!
• Quick-and-dirty taxonomy of existing open source tools. What are the toys in the toybox? Make some opinionated choices on our starting place.
• Want to have something to show at EGU Vienna in May 2025. AGU 2025, Fernando wants to be able to make noise (proposal in Spring for a Sunday workshop).
• .github repo for landing page.
• People want synchronous chat.
  • Slack / Discord / ?
• Decision record space!
• GitHub Discussions: Slower paced, Jupyter settled on Issues and PRs. discourse.jupyter.org is the main other place. Chris Holdgraf is currently admin, but Fernando isn’t. Will create a new geojupyter category in discourse!
• Create a GeoJupyter website!
• Dig into JupyterGIS codebase.
• DEADLINE: In 2 months, Fernando wants to know how I think JupyterGIS fits in GeoJupyter.
  • Talk w/ Anne Fouilloux with Simula - responsible for jupyterGIS user stories
• DEADLINE: 15th January; abstracts due for EGU! https://www.egu.eu/news/1165/call-for-abstracts-to-the-egu25-general-assembly-now-open/
  • Should have something ready to show in March.
• DEADLINE: Spring 2025; submit AGU Sunday workshop abstract.
• DEADLINE: Spring 2025; propose AGU GeoJupyter session
• Before EGU: Spring 2025; working meeting in Boulder, CO?
• October 30th 2025: Funding proposal!
  • Fernando meeting with NSF Monday to pitch the idea for funding. No pressure on Matt for this at the moment!
• Half hour check-in every two weeks!
• Another important community: CryoCloud! Work there! Share prototypes, get input!
  • Prioritize Openscapes and CryoCloud
• 2i2c serves other NASA communities, they can help liaise
• Eat our own dogfood ASAP!

Misc

• In our space, the output is almost always a map. Unlike Notebooks, where the output is more free-form / general
• Jupyter: Thinking and Storytelling With Code and Data - “how do we think about narrative in the context of computing in Jupyter”
  • Re-imagine the storytelling aspect; but the centerpiece of the story is the map!
  • Fernando: Read this! Talk to people!
  • See ArcGIS story maps
    • <find.wilderness.org>
• How to combine/compose existing tools like Mapbox/MapLibre, QGIS, …?
• Kevin: The analysis part is where people feel the need to fire up QGIS.
• Brookie: Bi-directional interaction with map through code. A plugin or jupyter notebook cell, write code that affects the map. Attach an object to a code cell reference. E.g. draw a box on the map and then interact with the stuff inside the box in the code.
  • Fernando: Mayavi Python 3d viz library. More user-friendly API for C++ mapping tool <___>. Fluid interaction, e.g. select something on the map and assign it to variable x! Then you can act on x in your code. Map adapts when modifying x
  • ⭐ CartoDB (now called Carto) had a neat trick for styling their online maps. Created a CartoCSS format. You write CSS to change the color of roads, line width, etc.
    • How to get people who can code a little (CSS, JSON) to learn a more modern way of working without them realizing it so much.
  • Brookie: Imagining a Map object, instantiated. Has a bunch of autogenerated names that all begin with _. If you change one of those autogenerated names, e.g. to "restaurants", you can do map.restaurants.
    • Like Kevin’s hack of reusing existing syntax
    • Andrew Bray: Deep in Quarto ecosystem, also cares deeply about storytelling. Showed Fernando some “scrollytelling” articles from NYT. Andrew is working on how to do that in Quarto!
    • Ed Tufte - Yale prof written classic books on data viz. One of his examples is Menard’s diagram on Napoleon’s march on Moscow. Andrew contacted Tufte to make a scrollytelling version of this map. Dynamically shows the same data over time.
• AGU got funding from Sloan in 2023 for Notebooks Now. To bring notebooks into the AGU peer review publication process. Funding finished, but project incomplete.
  • Involves adding to MyST authoring pipeline.
  • Fernando very interested in seeing this.
  • Wants Jupyter as part of the official scientific record for publishing. Standard process.
  • Audited MyST/AGU work for openness. Carthic(?). CurveNote. Hope Carthic will provide more funding. (Sorry, this isn’t clear because I got behind on notetaking!)
  • AGU & physics communities leading the way.
  • Want scrollytelling in MyST!