Organize & move your UDFs as you build your workflow on a free form canvas, allowing you to build analysis & workflows without being limited to a linear flow!
Fused Canvas still runs individual serverless UDFs but you can now more easily connect them together
Share your work to anyone with a single click
Try out Fused Canvas right away for free
]]>Last year Chris showcased how Pacific Spatial uses Fused to analyze 100 Billion drive records. Pacific Spatial Solutions is a company that provides geospatial analysis & data science services for various industries in Japan.
River plumes carry nutrients/sediment to the ocean and are split in 2 broad categories:
Mapping them can be helpful for understanding the health and human activities impact on the ocean.
These datasets are open and available through the Jaxa Earth API :
Traffic accidents and traffic safety has been one of the most important problems for the last many decades. Now that we have publicly accessible data regarding the accidents, we need to find ways to access and analyze it.
The Japanese National Police Agency published yearly traffic incidence data all around the country.
Explore the code directly in Fused:
Japanese summers can get very hot, which is amplified in large urban areas creating urban heat islands.
Prolonged exposure to the sun can cause dehydration and sunstroke, so we want to give Tokyo residents and tourists an app that can help them navigate the city in a more comfortable way.
Pacific Spatial Solutions is creating shadow maps for the city of Tokyo by combining 121 point clouds in batch and transforming them into 0.5m resolution DSM data and dynamically rendering them in Fused:
The way we interact with data changes constantly; we only need to look at the last few years to see that we are seeing a shift as part of the "Great Decoupling” – a broader movement toward zero-infrastructure.
By adding Serverless Compute – via cloud functions or the browser – to formats like Apache Parquet and engines such as DuckDB, we unlock blazing fast analytics without clusters or VMs. Now AI is entering the mix, and we think this is the perfect timing to put all the pieces together.
We don’t think analysts, data engineers & scientists are going anywhere. We believe that “AI won’t replace you, a human using AI will”. AI amplifies the creativity and speed of the people already doing the work.
Here’s how we think the future of analytics looks like:
A few months ago Andrej Karpathy coined the term Vibe Coding: letting LLMs loose and just give in to whatever code suggestion comes around, to quickly get a prototype of a new project, specifically for web development.
We’ve since seen the term Vibe Analytics starting to pop up around, applying a similar approach but to analytics: leveraging LLMs to make it easier and faster to create analysis, make dashboards and share them to the rest of the team.
There are a few core differences though:
| Vibe Coding | Vibe Analytics | |
|---|---|---|
| Context | None, starting from scratch | Rooted in existing data |
| Output | Code, experiments | Live insights, dashboards, APIs based on real datasets |
| Guidance | Fast loops via UI | Human-in-the-loop workflows |
| Purpose | Rapidly creating app prototypes | Quickly answering complex data questions |
Help us build the future of data analytics. Reach out to us directly (info@fused.io) or see our open positions.
We did take inspiration from Andrej Karpathy's recent Software is Changing (Again) talk at AI Startup School, as you may have noticed from the title.
]]>At Fused, we want to allow all data teams to get work done quickly. We want to make sure to do this on top of a secure and compliant platform.
Through the comprehensive auditing process overseen by Insight Assurance, we've demonstrated our adherence to the stringent requirements outlined by the SOC 2 Type 1 standard, reinforcing our dedication to safeguarding sensitive data and maintaining operational resilience.
Request the full report by email: info@fused.io.
We're actively pursuing SOC 2 Type 2 certification, building on our existing Type 1 compliance.
]]>Our mission is to help data teams get stuff done quickly, which is relevant for the people we talked to at EO Summit.
Here are some of our takeaways:
A lot of us are going through a bit of an existential crisis, while realising that damn, yep, AI can help us write code to glue datasets together quickly. Knowing the intricacies of a Python library isn’t a competitive advantage for individuals and companies to build the best analytics anymore.
That’s all well and good when working on a small, one-off project. But conferences like EO Summit keep showing that there’s more data than ever before. Archives of imagery, 3D point clouds and any dataset keeps growing in scale, resolution & time backlog. Problems aren’t always local, nor limited to a small time & place.
AI is changing what it means to build software & products, there’s not doubt about that. But executing it at scale is still a dark art.
At another conference, the Cloud Native Geopaptial last month Brianna Pagán shared her story of the Los Angeles fires taking down her home, and while having so much data available, so little was actually accessible in a helpful manner quickly. This topic came again at this conference, wildfires being the prime example of the complexity of making timely and updated use of data in times of need despite having so much.
A few months ago, our own Milind Soni made a quick interactive dashboard in a few hours as a proof of concept of what rapid tools could look like for wildfire updates using Fused.
These conversations are directly going into our internal product development discussion; as we write this, we're developing tools to build these dashboard for rapid iteration. Stay tuned for more on that soon!
Nadine Alameh, at the head of the Taylor Geospatial Institute & former CEO of the OGC, the consortium in charge of standards for all things geospatial, mentioned the lack of companies in the middle solving problems between data providers and companies building analytics products. This is something Aravind, head of Terrawatch -who organised this conference- has been saying for a while.
That’s why we were excited about being at EO Summit, we think we can make a difference tackling these problems!
We’re building tools that make getting data from all types of places & formats together, execute code (however it was written) at any scale with a few lines of code and sharing it all to whomever needs it, be it interactive dashboard, CSVs or tile servers.
If you’d like to learn more about Fused, book some time with Max, our Developer Advocate right here to get a demo!
]]>We worked with Emma Quirk (Senior Data Analyst) and Majid Alivand (Senior Data & Analytics Manager) to showcase how Fused can help bring all these datasets together. In this webinar they give an overview of the industry challenges interfacing backend data analytics with frontend data consumption. Emma walks through the notebook she used to model climate and irrigation patterns for vineyards.
Building useful & actionable weather models for environmental insights requires bringing datasets from:
To address these challenges, Emma & Majid used Fused + H3 to bring together datasets like GridMET climate, CDL crops, LANID irrigation, and gSSURGO soils by converting them from raster to H3 to build unified parquet files.
By building UDFs to manipulate each dataset, the team can iterate on their analysis in seconds while H3 allows them to more easily combine all the datasets together.
Try out the notebook from the presentation for yourself:
Sign up for a free Fused account and try it out for yourself!
]]>At Fused, we’re building tools to help data scientists work more efficiently: we want to give them the ability to work on any dataset, create an analysis and scale it to the whole world with just a few lines of code. But data scientists don’t work in a vacuum, and analysis aren’t (always) done because people are simply curious about a topic.
We built Fused Apps in the spirit of allowing a single person to do all the work themselves; and in this first episode of Inside Fused, a series of blogposts about how we’re building Fused, we want to take you behind the curtain to show how Fused Apps is built.
Fused provides both a Python package to run User Defined Functions (UDFs), and Workbench, a browser-based IDE to write, execute, visualize them as well as create Fused Apps to make interactive frontends.
Here’s what Fused Apps look like:
Fused Apps. From left to right: the list of apps that have been loaded in Workbench, the app code editor, and the running app itself.
The App code editor & renderer allow users to write their own Python code using Streamlit to build a frontend entirely in Python, a language most data scientists already work with.
We want data scientists to be able to go from “hey, that’s a cool idea” to “here’s what it looks like” without tech getting in their way. Especially in a world where LLMs make writing code simpler, the bottleneck becomes the speed at which data scientists can execute & ship code, not write it.
Fused Apps offers a way for data scientists to orchestrate their entire workflow using Python, without having to worry about backends, scaling, or clusters. Fused Apps complement our Map Builder, which offers a way to build data processing and backend functions, by offering an end-to-end workflow. At Fused, we use this for ingesting and managing datasets, managing resources we make available to our UDFs, and finalizing analyses.
Fused Apps work on any device, as long as there's a browser!
In short, Fused Apps tie together a frontend application for users to write Python code with Streamlit, a backend that saves & serves these apps and provides shareable links, and a product experience tying all of this together (error handling, autocomplete suggestions, async UI functionalities, etc.)
Fused Apps are built on top of Stlite & Streamlit. Streamlit being a library allowing you to write a frontend application, entirely in Python, and Stlite an in-browser version of Streamlit. This allows people familiar with Python but not so much frontend development (like data scientists) to have something rendered on screen in HTML, but only with writing Python.
Stlite provides the ability to run apps in the browser, and while Stlite Sharing does support URL-encoded mechanisms for app sharing, the URL is the same as the code, preventing users from updating said code; there’s also no such thing as app catalogs, etc. So we ended up using Stlite as the engine and built a product experience around that.
The frontend for building a Python-in-the-browser application already exists, with Streamlit & Stlite. However the backend had to be built from the ground up.
Originally, we didn’t even have a way for people to save their apps! Our internal workflow while developing & testing this was to have a Slack channel where we pasted our app links in to be able to find them later on
This is at the the core of our development philosophy: Build a prototype, use it a bunch, find the pain points, fix them, build more. In this case, saving & tracking apps was the next piece to build.
This is how we added functionality like Github integration, sharing options, and an app catalog. All these were only added after we got some traction internally and from early customers.
Sharing Fused Apps is just a couple of clicks now
The next piece of this puzzle is to realise that running Python locally on your machine and in the browser is quite different. We want to make the experience for the user as seamless as possible: we’re building products for data scientists to build everything in Python.
We use Pyodide to run Python in WASM. This enables the same Python language to be used in a new environment – the browser. The browser environment is key because it gives us a way to safely ship applications to users.
Safely shipping software to people wasn’t always the simplest. Previous efforts via Flash and Java applets enabled a generation of rich web content. The promise of Java applets was run anywhere – that the same code could run on anyone’s computer. These technologies died out mainly because of security model problems. (Java lives on in Blu-ray disks.)
At the time of writing this, in mid 2025, browser applications are considered well sandboxed (with the possible exceptions of RowHammer/Spectre type issues), browser applications open when you want them, and go away completely when you dismiss them, all the while performing more and more complex tasks. Browser applications handle video encoding/decoding for video conferencing, graphics rendering for games, maps, and design, and more frequently, for programming. Companies like Figma are building complex, professional grade applications for the browser first.
From a development perspective, the browser becomes an operating system. It becomes less important to know what exact hardware & underlying operating system the users have, and we develop applications for the browsers’ APIs. Those applications are inherently portable to new devices, because the heavy lifting is porting the browser to these new devices.
There are other projects out there that allow building hosted Stlite applications, however these miss some of the features that people expect from a well rounded product. Here are few things we’ve added to Fused Apps:
fused and pyarrow pre-installed)Beyond the technical challenges we want to build a product that helps data scientists build, iterate and ship faster. A data scientist can build an analysis that takes in different parameters and make an interactive graph for their project manager to test out directly all without needing support
A big part of this is the philosophy of how we build things at Fused: We’re a team of engineers & data scientists. A lot of the features we’re listing above here come directly from our own usage of Fused Apps while making real applications with some of our customers.
We care about how fast it takes for a new user to click on a Fused App and start running it, or what the experience of saving & sharing an app looks like.
Every Fused App comes with some packages pre-installed, such as fused and pyarrow, which are helpful for data scientists. (Streamlit comes with a number of other common packages like Pandas and NumPy already.) But this comes at the expense of loading time, as each new app is a self-contained application which requires downloading all the required packages. This is leading us to spend time optimizing the initial loading time of Pyodide.
Fused Apps, and any implementation of Python in the browser, isn’t as customizable as a fully local setup. Packages with native code that are not prebuilt for WASM will not work. At the time of writing, some popular packages like Pandas & Numpy are built and supported, but others aren’t. The backbone of the map processing pipeline, GDAL, for example, isn’t currently supported. This isn’t from lack of popularity, but reflects the complexity of building GDAL.
Some packages will need architecture updates. As another example people currently call into ffmpegfrom Python using the CLI. The WASM environment does not have a CLI concept, and this would need to be replaced with library calls. Other packages which are not well suited to this architecture (such as Torch) might have alternatives developed because the wasm ecosystem is so attractive a development target.
An example of this is transformers.js.py. This library (developed by Yuichiro Tachibana, also behind Stlite) needed more than one level of architectural adaptation to get running in the browser, but once adapted it brought the capability of running lightweight LLMs. This allows us to ship applications that run small LLMs models, running locally in the browser!
We made a Fused App exploring the USDA’s Crop Data Layer (CDL) dataset, a dataset of different crop types in the US. Instead of showing all 130+ categories, we can just take a text input, run a similarity analysis on the fly against all the categories and find the closest CDL crop type!
You too can find out what Bunnies prefer by trying this Fused App here
This is a fast changing ecosystem though, for example we’ve made a small contribution of a build of the H3 package specifically because we wanted to have H3 supported in Python in the browser. The list of supported packages built in Pyodide is growing quickly.
Moving large amounts of data in and out of the browser is slow, so data-intensive libraries like Torch or Dask are not well-suited for this. We solve this by bringing in the compute performance & flexibility of Fused UDFs, which run native (non-WASM) Python in a cloud environment. Fused UDFs run code much closer to the data, improving performance, and are not limited by what packages are available for Pyodide, improving flexibility.
Install times for Pyodide are relatively long from a user point of view, as the application isn’t stored locally and installed each time it's opened. This will most likely keep improving over time as the technology is developed further. For example Pyodide wants to add memory snapshotting to help with this, but it isn’t stable yet.
Even when the application is stored locally, we found we needed to reinitialize Pyodide in some cases. There is no concept of switching virtual environments in Pyodide, since running a new script reuses the same Python VM. When switching between apps in Fused, we reinitialize Pyodide in order to prevent packages from one app interfering with another. If we didn’t do this, a user might accidentally rely on a package installed by App A in developing App B, which would then not work when sharing App B.
We also needed to be careful with when the app can run, since the app has access to the user’s browser context. We chose to give users the chance to inspect the app’s code before running it if it would have access to their Fused account.
Code written in Stlite and Pyodide looks almost the same as regular Python, but there are slight differences. Many of these come in the form of adapting synchronous and asynchronous code. Stlite for example allows for top-level await because the browser’s event loop is being used. This can be tricky to work with because regular advice for working around asynchronous code in Python does not work with Pyodide.
Async-aware UI allows us to provide improved feedback as an app loads a light weight LLM model or data to display
In order to create a good product experience, we added our own syntax checking and linting on top of Pyodide.
Lots of small features go a long way to delighting users with a smooth experience.
Fused Apps are part of the free tier of Fused. You can use Fused Apps without logging in at all, and with a free login you can save and share your apps. We want to take a moment here to explain why this is naturally a free offering and will continue to be free.
As a browser-based application, all of the code execution and data transfer happens in the user's browser. This means that we do not need to sandbox code execution or pay for cloud compute resources to run anyone’s Fused Apps.
Where we do incur costs are in the shared control plane layer of Fused. Technically, the control place layer doesn’t see much difference between an app and a UDF. As a result, the incremental cost of serving an app user is very low and it is easy for us to offer that for free. The core offering of Fused is the backend serverless execution of code, which is our paid product.
Don’t take our word for it, give Fused Apps a try for yourself! As we mentioned, Fused Apps are free and don’t require login. You can check out our catalog of public Apps in Fused Workbench.
Recently we announced re-partitioning the Crop Data Layer dataset into H3 hexagons for anyone to use and hosted the resulting dataset on Source Cooperative.
Alongside this we created a public Fused App allowing you to explore any crop for the 2024 dataset
We firmly believe data scientists need tools that give them the independence to do their work rather than asking for support to scale their analysis or share their results.
We need smart people to help us build all of this. We are hiring for:
If you’d like to join the team, send us your info here!
Join the team!
]]>Watch the announcement video:
Fused Apps are one of the the fastest way to write Python, directly in the browser.
As soon as you save your Fused App, you can create a shareable link for anyone to open
Fused Apps are built on top of Pyodide, Streamlit & Stlite but offer many features:
transformers.js.py
We've curated a catalog of existing Fused Apps to get you started, including:
You can try out Fused Apps for yourself directly in Fused Workbench and read our dedicated Docs page
We wrote a dedicated technical blog post going into details of how Fused Apps are built:
On Thursday May 22nd we'll be hosting a webinar showcasing how our customers use Fused
At its core, Sylvera rates carbon projects. Our ratings are powered by several earth observation and geospatial analysis data products. From climate risk data, and deforestation indicators, to biomass-predicting ML models, a wealth of data goes into generating a single-letter rating.
With such a wealth of information, it's challenging to determine how best to present insights to the client. On one hand, we want the client to be able to see the data that tell the story of the project's rating. On the other, we don't want to overwhelm them with fun interactive visualizations that leave them without a clear takeaway.
This is why we decided to organize an internal hackathon, aimed at discovering what the modern geospatial data stack can do when it comes to enabling user experience with geospatial data.
To enable quick iteration, selecting the right technology stack was crucial. Our data lives in Zarr format in Arraylake, a data lake platform for managing array data.
We explored the current landscape and considered several options: Should we serve pre-computed visualizations through xpublish or TiTiler? Create interactive dashboards through Felt, or build them in Mapbox?
Ultimately, we chose the most agile and powerful stack: an interactive React + deck.gl application powered entirely by Fused.
We created a React dashboard, where each element renders the output of Fused UDFs. This required an entire DAG of UDFs.
The core element is the Data UDF, which fetches raw data for a given area of interest. The subsequent UDFs then process this data for the application.
In the example below, The Visualisation and Data Overlay UDFs prepare a web map. The Histogram UDF uses the hammer-and-nails pattern to get the distribution for each year. The Timeseries UDF computes the average per year. This setup is highly modular and makes it straightforward to present the same data in a myriad of different ways.
This is the custom function to create a layer from a Fused HTTPS endpoint called as a File which returns a .png.
const createFusedFileLayer = (
layerId: string,
fusedId: string,
bounds: [number, number, number, number],
year: number
) => {
const key = `${layerId}-${year}`;
const param = `year=${year}`;
const imageUrl = `https://www.fused.io/server/v1/realtime-shared/${fusedId}/run/file?dtype_out_raster=png&${param}`;
const layer = createBitmapLayer(key, imageUrl, bounds);
return layer;
};
import { createRoot } from "react-dom/client";
import DeckGL from "@deck.gl/react";
import { MapView } from "@deck.gl/core";
import { BitmapLayer } from "@deck.gl/layers";
import { TileLayer } from "@deck.gl/geo-layers";
const BOUNDS_AFRICA: [number, number, number, number] = [
-25.35, -46.95, 51.35, 37.35,
];
const UDF_H_AOI_FILE_CALL = "FUSED_UDF";
function createTileLayer(id: string, data: string[]): TileLayer<ImageBitmap> {
return new TileLayer<ImageBitmap>({
id: id,
data: data,
maxRequests: 20,
maxCacheSize: 200,
pickable: true,
highlightColor: [60, 60, 60, 40],
minZoom: 0,
maxZoom: 19,
tileSize: 256,
zoomOffset: devicePixelRatio === 1 ? -1 : 0,
renderSubLayers: (props) => {
const [[west, south], [east, north]] = props.tile.boundingBox;
const { data, ...otherProps } = props;
return [
new BitmapLayer(otherProps, {
image: data,
bounds: [west, south, east, north],
}),
];
},
});
}
function createBitmapLayer(
id: string,
image: string,
bounds: [number, number, number, number]
) {
return new BitmapLayer({
id: id,
image: image,
bounds: bounds,
pickable: true,
highlightColor: [60, 60, 60, 40],
minZoom: 0,
maxZoom: 19,
});
}
const createFusedFileLayer = (
layerId: string,
fusedId: string,
bounds: [number, number, number, number],
year: number
) => {
const key = `${layerId}-${year}`;
const param = `year=${year}`;
const imageUrl = `https://www.fused.io/server/v1/realtime-shared/${fusedId}/run/file?dtype_out_raster=png&dtype_out_vector=csv&${param}`;
const layer = createBitmapLayer(key, imageUrl, bounds);
return layer;
};
const createBasemapLayer = () => {
return createTileLayer("basemap", [
"https://basemaps.cartocdn.com/light_all/{z}/{x}/{y}.png",
]);
};
export default function App() {
const bounds = BOUNDS_AFRICA;
const year = 2000;
const basemap = createBasemapLayer();
const fused = createFusedFileLayer(
"fused",
UDF_H_AOI_FILE_CALL,
bounds,
year
);
return (
<DeckGL
layers={[basemap, fused]}
views={new MapView({ repeat: true })}
initialViewState={{ latitude: 0.34211, longitude: 15.151583, zoom: 2 }}
controller={true}
></DeckGL>
);
}
export function renderToDOM(container: HTMLDivElement) {
createRoot(container).render(<App />);
}
The application we built isn't yet fully featured to be put in front of users – but that's the point. We were not aiming for a finished product yet. Instead, we achieved rapid iteration that enabled us to gather relevant stakeholder feedback.
The speed we could reach wouldn't have been possible without Fused's development platform. Fused unifies three traditionally separate stages — prototyping, scaling, and visualization—into a single seamless solution. Thanks to this, Fused was an indispensable tool for product iteration.
In the future, we would like to explore the coming integration with Zarr stores. Being able to not only visualize the results but also to immediately persist them into a Zarr store will be a game-changing capability for anyone who uses Zarr as the persistence layer.
]]>These 2 datasets are both available on Source Cooperative for anyone to use, free of charge!
We've taken 2 popular datasets and made them available in H3 Hexagons, this provides a number of benefits:
You can read more about H3 Indexes on the dedicated page.
We're providing:
[2012, 2014, 2016, 2018, 2020, 2022, 2024]hex7 and hex8 resolutions2020hex7 and hex8 resolutionsThese datasets are completely free for anyone to use, with or without Fused. To showcase what's possible we have made a public Fused UDF anyone can use without an account to explore these.
You can try it here without any account.
]]>
Fused is excited to be featured in a TED Talk today by Peter Wilczynski, CPO at Maxar, which explored a multi-source site monitoring scenario of Vancouver's greening initiatives over the past five years powered by the Fused platform.
Site monitoring scenarios are at their most effective when data from disparate sources is combined—from satellite imagery and map data to municipal and demographic data. Analytics today requires bringing all these different sources of data together: from raw pixels to contextual intelligence — in real-time, seamlessly, and at scale.
This is an iterative process, each dataset being in its own format & hosted on different cloud providers, updated independently and on different schedules.
Fused is built by data scientists, engineers & open source contributors who know the reality of building pipelines with breaking changes in libraries, new file formats that require rethinking how data is ingested and business requirements that constantly change.
We firmly believe we need tools that help us bring as many datasets together as possible, especially in a world where LLMs and AI can increasingly talk directly to data.
If you’d like to try out an instance of the Fused workbench with pre-loaded data from Maxar later this month, which Peter demoed on screen, sign up for the waitlist on fused.maxar.com to get access and see for yourself this is more than a pre-recorded tech demo!
]]>We're launching Fused AI Builder, a simple way for you to give an LLM access to any UDF you want
MCP (Modal Context Protocol) Servers allow Large Language Models like Claude or ChatGPT to directly make HTTPS queries. This means they can now access APIs and make all sorts of calls.
But a lot of the internet still doesn't have APIs, be it a new dataset you just discovered that's hosted on an FTP server or data on an S3 bucket that needs some processing before it's useful.
That's where Fused UDFs come into play. UDFs allow you to write Python code that can be executed from anywhere, and more importantly called from anywhere.
By using UDFs with MCP Server you can now easily write any Python code and allow a LLM like Antrophic's Claude to execute it & use its output.
You can now use LLM + Fused UDFs to do a whole host of tasks:
🚧 This section is under construction. Our non-AI Agents are still working on this 🚧
fused-py!
Collections now allows you to organize your UDFs together as you work on different projects, save them together and keep your editor focused on the project at hand.
Collection is still in early access so you need to enable it under "Preferences -> Enable UDF Collections [Beta]" to access it
UDF Editor now comes with a host of new features & redesigned UI:
We've moved all the tools & information you need to easily share your UDFs into a dedicated page (and button). You can easily:
fused-pyOur Python library fused-py is getting some updates to make processing data in Python simpler, from a small one time task to processing huge datasets
fused.types to only having 2 simple types: fused.types.Bounds and fused.types.Tile.bbox object with bounds: a more generic term to pass geometries to UDFsfused.submit() for simple, multi job runsWe're introducing fused.submit() as a simple way to run many UDFs all at once in parallel
This can significantly speed up parallel tasks like:
Under the hood we've significantly improved how Fused caches results of recurring UDFs & cached functions + we've introduced new tools for developers to control caching:
cache_max_age argumentRead our Changelog to see every change happening with Fused 2.0
]]>The Global Entity Reference System (GERS) is a framework that structures, encodes, and matches map data to a shared universal reference within Overture. GERS helps organizations identify and reference their own datasets with standard identifiers to Overture data to help unify datasets.
In this blog post we show how to create simple endpoints with Fused UDFs to enrich a dataset with GERS IDs. We'll use the Overture Building footprints to first enrich a polygon with GERS IDs then another one to look-up metadata for a specified GERS ID.
When third-party dataset is spatially matched to an Overture feature it's "enriched" with that feature's GERS ID and becomes "GERS-enabled". This makes it easy to associate it by ID to any other GERS-enabled dataset.
gers_id buliding_name
0 08b2a1072534cfff020018b8a6efde22 James A. Farley Building
1 08b2a100d2cb6fff02000821de8bdff1 Pennsylvania Station
For example, a municipal government with a dataset of building footprints for local offices, coffee shops, and museums could match those entities to a GERS ID. This would enable the government to easily join its data other "GERS-enabled" datasets to enrich them with additional information such as insurance data, historical property values, restaurant reviews, fire risk, or rooftop solar potential.
We can create a UDF that takes in a polygon and returns a GERS ID. This UDF will spatially match the polygon to Overture Buildings and return the GERS ID of the building that intersects the polygon. This is useful for enriching a dataset with GERS IDs.
Users can design and preview the workflow interactively, allowing them to test assumptions and visualize their effect. Parameters can be adjusted, and the output can be previewed before running the UDF on the entire dataset.
import json
aoi = json.dumps({"type":"FeatureCollection","features":[{"type":"Feature","properties":{},"geometry":{"coordinates":[[[-73.9945443706057,40.751205998161026],[-73.9945443706057,40.749882643934455],[-73.99222917395902,40.749882643934455],[-73.99222917395902,40.751205998161026],[-73.9945443706057,40.751205998161026]]],"type":"Polygon"}}]})
@fused.udf
def udf(bbox: fused.types.TileGDF=aoi):
import geopandas as gpd
utils = fused.load("https://github.com/fusedio/udfs/tree/e1fefb7/public/Overture_Maps_Example/").utils
# 1. Convert bbox to GeoDataFrame
if isinstance(bbox, str):
bbox = gpd.GeoDataFrame.from_features(json.loads(bbox))
# 2. Load Overture Buildings that intersect the given bbox centroid
gdf = utils.get_overture(bbox=bbox.geometry.centroid, overture_type='building', min_zoom=10)
# How many Overture buildings fall within the bbox centroid?
print("Buildings in centroid: ", len(gdf))
# 3. Rule to set only one GERS on the input polygon
bbox['id'] = gdf.id.values[0]
return bbox
With Fused, it's easy to turn your UDF into an HTTPS endpoint. This enables you to run the UDF it programmatically via HTTPS requests to integrate the functionality into various workflows and applications.
This endpoint runs a public UDF with the code above. You can call it with a geojson of a single polygon in the bbox query parameter and it will return a geojson with the polygon and an assigned GERS ID.
https://www.fused.io/server/v1/realtime-shared/UDF_Overture_Buildings_GERS_Enrich/run/file?dtype_out_vector=geojson&bbox={"type":"FeatureCollection","features":[{"type":"Feature","properties":{},"geometry":{"coordinates":[[[-73.9945443706057,40.751205998161026],[-73.9945443706057,40.749882643934455],[-73.99222917395902,40.749882643934455],[-73.99222917395902,40.751205998161026],[-73.9945443706057,40.751205998161026]]],"type":"Polygon"}}]}
We can also create a sample UDF to do a reverse operation: look-up a Building and its attributes by passing a GERS id. A user will be able to pass a GERS id and the UDF will look-up the building and return its geometry along with attributes about the building.
To do this, we create a UDF that takes in a gers_id parameter. Because the first 16 digits of GERS correspond to an H3 cell, we can use the ID to create a polygon to spatially filter the dataset. It'll bring up any buildings that intersect the H3 cell. Once we have the building, we can easily work with its geometry object and attributes using GeoPandas.
@fused.udf
def udf(gers_id: str='08b2a100d2cb6fff02000821de8bdff1'):
import h3
from shapely.geometry import Polygon
import geopandas as gpd
import pandas as pd
utils = fused.load("https://github.com/fusedio/udfs/tree/e1fefb7/public/Overture_Maps_Example/").utils
# 1. H3 from GERS
h3_index = gers_id[:16]
print('h3_index', h3_index)
# 2. Polygon from H3
bounds = Polygon([coord[::-1] for coord in h3.cell_to_boundary(h3_index)])
bbox = gpd.GeoDataFrame({'h3_index': [h3_index], 'geometry': [bounds]})
# 3. Load Overture Buildings
gdf = utils.get_overture(bbox=bbox, overture_type='building', min_zoom=10)
# 4. Subselect building
gdf = gdf[gdf['id'] == gers_id]
# 5. De-struct the names column
normalized_df = pd.json_normalize(gdf['names'])
gdf = gdf.reset_index(drop=True).join(normalized_df)
return gdf[['id', 'primary', 'subtype', 'class', 'geometry']]
Here's how you can create and use an HTTPS endpoint for your GERS building lookup UDF.
This endpoint returns a CSV table of the building's GERS ID, primary name, subtype, class, and geometry. You can use this endpoint to enrich your dataset with GERS IDs by calling it with a GERS ID query parameter.
https://www.fused.io/server/v1/realtime-shared/UDF_Overture_Buildings_GERS_Lookup/run/file?08b2a100d2cb6fff02000821de8bdff1&dtype_out_vector=csv
For example, you could call this endpoint from a Google Sheet to enrich a dataset with GERS IDs. This sample Google Sheet returns the enriches the "primary" name column for any given Building GERS. Just drag the formula to apply it to any row below. It works by calling the "GERS lookup" endpoint with a GERS ID query parameter.
]]>Fused has been working with the team at The Overture Maps Foundation to enable direct access to their data through Fused UDFs. We are excited to share that the Overture docs now show examples on how to see how to integrate any Overture data into your workflows using Fused.
Overture Maps aims to provide foundational building blocks of data across various themes designed to be broadly applicable across industries. Our goal at Fused is to make it easy to work with Overture data and adopt standards (such as GERS). To this end, we are creating easy abstractions to access data, tools to perform foundational operations such as conflation and enrichment, and example workflows to inspire and help you understand how to leverage this data.
One of the key usecases for Overture + Fused is enriching datasets with Overture Maps data. This tutorial showcases 2 simple Python workflows that do this by performing a spatial join with Overture Buildings. This example of a simple enrichment operation will help you understand how to work with Overture data in your own workflows.
To follow along, check out the:
The Overture Buildings dataset is dividen into themes. Two key themes are:
We'll first show how you can load Overture data by reusing an existing Fused UDF, then write a User Defined Function (UDF) with custom logic to perform enrichment with a spatial join. You'll be able to run the resulting UDF for any custom area of interest (AOI).
Fused has a catalog of pre-made UDFs you can easily copy and repurpose for your own data analysis workflows. In the catalog, you'll find the Fused Overture UDF which enables you to quickly load Overture data from any of the themes for an area of interest (AOI). You can run the UDF with fused.run and specify an AOI to load data for using the bbox parameter. You may also pass optional parameters to select between Overture releases, themes, and columns - that way you can fetch only the data you need. In this example, we can specify the 'building' theme by setting the overture_type parameter.
import fused
bounds = [-73.9847, 40.7666, -73.9810, 40.7694]
fused.run('UDF_Overture_Maps_Example', bounds=bounds, overture_type='building')
The output should look like this:
id geometry class ...
24134 08b2a100d65a6fff0200b45ce7e2b99b POLYGON ((-73.98552 40.76736, -73.98557 40.767... apartments ...
24135 08b2a1008b259fff02007917db1c32d3 POLYGON ((-73.98441 40.76703, -73.98431 40.767... apartments ...
24178 08b2a100d6516fff0200ded2bf849c8a POLYGON ((-73.98375 40.76693, -73.98381 40.766... apartments ...
24179 08b2a100d6516fff02006dc174022a7e POLYGON ((-73.98346 40.76623, -73.98327 40.766... commercial ...
24180 08b2a1008b248fff0200315983940aa8 POLYGON ((-73.98407 40.76749, -73.98402 40.767... None ...
By browsing the UDF's catalog page, you can see its code and even copy it to run it interactively on the Fused Workbench. You'll notice that the UDF uses the get_overture helper function to read from spatially partitioned parquets of the overture data releases, hosted in a Source Cooperative S3 bucket. The source code of the helper function is fully open and hosted on GitHub here.
Here's a simplified version to show the core of what's going on in get_overture. It constructs the table path on S3 and then uses the table_to_tile helper function from Fused to load data that falls within the specified bounding box. This approach allows you to efficiently perform spatial queries on a large dataset and load only the records within the given area.
# Structure the table path with input parameters
table_path = f"s3://us-west-2.opendata.source.coop/fused/overture/{release}/theme={theme}/type={overture_type}"
# Load the data within the bounding box
df = utils.table_to_tile(bbox, table=part_path)
The example above shows how to run an existing UDF to load Overture data, but it's likely you want to write your own data transformations. You can borrow get_overture to load data into your own UDFs. As an example, here is a UDF to load Overture Buildings polygons and Overture Places points. The UDF perform a spatial join between them to determine which points fall within each building.
@fused.udf
def udf(bbox: fused.types.TileGDF = None):
utils = fused.load("https://github.com/fusedio/udfs/tree/e1fefb7/public/Overture_Maps_Example/").utils
# 1. Load Buildings
gdf_buildings = utils.get_overture(bbox=bbox, theme='buildings')
# 2. Load Places
gdf_places = utils.get_overture(bbox=bbox, theme='places')
# 3. Create a column with the Buliding Name
gdf_buildings['primary_name'] = gdf_buildings['names'].apply(lambda x: x.get('primary') if isinstance(x, dict) else None)
# 4. Spatial join between Places and Buildings
gdf_joined = gdf_places.sjoin(gdf_buildings[['geometry', 'primary_name']])[['id', 'names', 'primary_name', 'geometry']]
return gdf_joined
To run this UDF, you simply call it with your AOI. Fused will execute the code with the given parameter then return the UDF's output.
import fused
import geopandas as gpd
import shapely
bbox = gpd.GeoDataFrame(geometry=[shapely.box(-73.9847, 40.7666, -73.9810, 40.7694)], crs=4326)
fused.run(udf, bbox=bbox)
This will return a GeoDataFrame with the geometry of the Place, the primary_name of the building it falls within, and other attributes as defined in the UDF's return statement.
The output should look like this:
id names primary_name geometry
3883 08f2a100d65160860308e7269804dcb7 {'common': None, 'primary': 'Alamo Rent A Car'... The Sheffield 57 POINT (-73.98404 40.76661)
3884 08f2a100d65160860306e05f63f3cbaf {'common': None, 'primary': 'National Car Rent... The Sheffield 57 POINT (-73.98403 40.76661)
3896 08f2a100d6516b110387be8074670bfa {'common': None, 'primary': 'Quality Fashion',... Hearst Tower POINT (-73.98370 40.76660)
3898 08f2a100d65147290359c345582a702e {'common': None, 'primary': 'House Beautiful M... Hearst Tower POINT (-73.98338 40.76664)
As a second example, you can also enrich the Overture Buildings dataset with metadata data from the National Structure Inventory (NSI) API. The NSI API offers point data on buildings in the U.S. that is relevant to hazard analyses.
This UDF loads the Overture and NSI datasets, performs a spatial join to enrich the building polygons with hazard metadata, and returns the enriched GeoDataFrame. It can be used within a larger analysis workflow to enrich building polygons to calculate risk indices. You can read more about performing spatial operations to enrich Overture Buildings with NSI in our geospatial processing guide.
@fused.udf
def udf(bbox: fused.types.TileGDF = None):
import geopandas as gpd
import pandas as pd
import requests
utils = fused.load("https://github.com/fusedio/udfs/tree/e1fefb7/public/Overture_Maps_Example/").utils
# 1. Load Overture Buildings
gdf_overture = utils.get_overture(bbox=bbox)
# 2. Load NSI from API
response = requests.post(
url="https://nsi.sec.usace.army.mil/nsiapi/structures?fmt=fc",
json=bbox.__geo_interface__,
)
# 3. Create NSI gdf
gdf = gpd.GeoDataFrame.from_features(response.json()["features"])
# 4. Join Overture and NSI
cols = ["id","geometry","metric","ground_elv_m","height","num_floors","num_story"]
join = gdf_overture.sjoin(gdf, how='left')
join["metric"] = join.apply(lambda row: row.height if pd.notnull(row.height) else row.num_story*3, axis=1)
return join[cols]
The output should look like this:
id geometry val_struct med_yr_blt ...
24178 08b2a100d6516fff0200ded2bf849c8a POLYGON ((-73.98375 40.76693, -73.98381 40.766... 378633.733 1939 ...
24178 08b2a100d6516fff0200ded2bf849c8a POLYGON ((-73.98375 40.76693, -73.98381 40.766... 348190.820 1939 ...
24178 08b2a100d6516fff0200ded2bf849c8a POLYGON ((-73.98375 40.76693, -73.98381 40.766... 378633.733 1939 ...
In this short tutorial, we outlined how you can integrate Overture data into your workflows using Fused. We saw how you can use Fused to load the data, write a custom Python workflow, and run it for a custom AOI. We hope the these foundational pieces help you see how you can unlock the full potential of Overture Maps and start creating your own workflows to enrich your own datasets.
]]>Pilot Fiber is a commercial Internet Service Provider primarily in New York City. Our primary value proposition in competing with national-scale ISPs is our commitment to customer experience–we are fast and flexible in responding to customer needs in all aspects of the business:
We use Fused to support all of these areas, and this post focuses on the last one: incident management.
One of the most common ways a customer's service can be impacted is through damage to the physical fiber cables connecting them back to a data center and the internet. Almost all fiber optic cables in Manhattan run through a shared manhole-and-duct system beneath the streets. As such, road construction or work by other providers in a manhole has the potential to damage the equipment of multiple providers. When that damage occurs, it is first come, first served to get your network repaired and customers back online. Field teams from multiple providers can't work in the same manhole simultaneously, so being onsite first and ready to repair can mean a difference of hours in customer downtime.
Because of this, Pilot uses an active fiber monitoring system across our network. Sophisticated devices in our data centers are constantly shooting light down the fibers in our network looking for potential damage. Those devices return a reflectance signature from the fiber and compare it with a reference "snapshot" created when that fiber was initially installed in a building.
When an anomaly is registered, it immediately fires an alert giving a fiber route and distance to the potential problem (i.e. "There is unexpected light loss on the fiber serving 1234 5th Avenue at a distance of 2.351 kilometers from the data center."). When this happens, our engineering and support teams analyze the data within minutes to determine the issue's exact location and, if necessary, get crews headed to the site to begin repairs.
Historically, this analysis has required the attention of an Outside Plant engineer with access to specialized software and network knowledge, regardless of the time of day or day of the week. This bottleneck is not ideal when time is of the essence, even at 3 a.m. So today, we are creating a more sophisticated future using Fused to make this information accessible to more support team members and make our response times even faster.
Using Fused as a back-end glue layer, we built a web app allowing users to select a route and distance and calculate where the system has registered the fault. We also created a simple user interface that provides the user a view of nearby network infrastructure and automatically generates the reports field crews would need to complete repairs based on that nearby infrastructure.
The workflow requires a series of calls to UDFs that act as intermediaries to a Postgres/PostGIS database, which in turn is sourcing data from other internal sources. This structure allows us to easily keep the business logic organized at the UDF layer while limiting the scope of data access and security via Postgres and internal processes maintaining the sync.
The basic process is seen below:
Workflow diagram.
Two separate flows are initiated when the user inputs a route and a distance to process. One retrieves the selected route to load onto the Mapbox-based map within the app, while the second kicks off a processing chain to analyze the fault information. This chain utilizes UDFs that assist in isolating the location of the fault and relevant nearby infrastructure and adding elements to the map display to assist the user in visualizing what may be occurring.
UDF to find fault location.
If you consider the cables you see strung along utility poles, they are not perfectly straight: they can sag, bend, go up and down, and have coils of extra cable along the way. The same is true of our cables under the streets. All of those variations add distance to the run, which needs to be considered when determining where a fault is likely to be located.
Taking those variables into account, we use Fused to apply GeoPandas and PostGIS spatial functionality to assess where the fault is most likely to be located. After calculating that location, the tool loads splice cases along that route that point to where problems are most likely to have occurred, and slack loops built into the route to make the user aware of nearby capacity that could enable faster repair of more significant damage. We next determine which splice cases are closest to the likely damage point and if any of those are within 150m of the automated distance calculation. These manholes would be the first locations our field teams would be sent to investigate.
Once we have determined the relevant nearby splice cases, we use another UDF to build a CSV that reproduces what a splice report export looks like from our primary fiber mapping software. This report details what cables enter the case, what fibers on which cables are spliced to which fibers on other cables, and what building and/or customer circuits are being carried along which fibers. Fused then integrates with a separate process we previously built to process that CSV into the final file that goes to the field teams. This gives them the confidence to quickly address issues while minimizing the risk of damaging other circuits, as well as giving them the information needed to monitor the status of other services in the case while they work.
The impact of Fused across this process is many-fold:
Pilot's success in the market is largely based on our flexibility and responsiveness to customer needs, which is never more important than when physical damage to the network is impacting their service. Within this scenario, Fused is providing a critical layer enabling us to offer more non-technical users access to data in multiple systems through a simple UI that will result in repair teams moving to restore service to our customers as quickly as possible.
Fused is an ideal product for Pilot Fiber in that we can increasingly make highly technical information available in a usable format to additional teams throughout the company in support of our drive to be fast, flexible, and accurate in delivering service to our customers in all aspects of the business.
]]>BlackPrint Technologies began its journey as a satellite mapping service designed to help municipalities modernize their property registries. Recognizing a greater opportunity, we transitioned into the private sector to address a significant gap in the commercial real estate market: the need for accessible, actionable data. Today, our platform empowers professionals with precise acquisition intelligence, transforming how decisions are made across Mexico and, soon, all of Latin America.
In this blog post, we will explore how BlackPrint built the backend of its intelligence platform with Fused to provide a comprehensive view of the commercial real estate market in Mexico.
Real estate professionals face fragmented data sources and manual processes. This complexity limits access to critical metrics such as zoning regulations, demographic patterns, and traffic trends, making site selection decisions both high-stakes and prone to errors. BlackPrint recognized this challenge and set out to create an all-in-one platform, transforming disjointed data into actionable intelligence to empower professionals across the commercial real estate landscape in Latin America. Our Solution: The BlackPrint Approach
BlackPrint's platform offers a comprehensive suite of tools and insights, including property and zoning data, demographic and socioeconomic analytics, and detailed foot and vehicle traffic analysis. Underpinning this solution is a staggering volume of geospatial data sourced from diverse datasets such as cadastral records, demographic studies, and traffic sensors. This vast amount of information is meticulously analyzed to deliver actionable insights, all while being presented through an intuitive and user-friendly interface. By simplifying complex data into accessible formats, BlackPrint empowers users—regardless of their technical expertise—to navigate and leverage insights with ease. Building such a robust platform required not only expertise but also the right tools. That's where Fused came in.
Fused became an indispensable partner in turning our vision for BlackPrint into reality. Its end-to-end cloud platform allowed us to move from concept to MVP significantly faster by simplifying our data processing and data delivery workflows. Before using Fused, we faced daunting challenges, such as processing and visually inspecting terabytes of traffic data and massive datasets from the Overture Maps Foundation for points of interest.
These tasks, which previously required extensive time and resources, became streamlined and efficient with Fused. Its ability to process geospatial datasets via https-accessible Python functions made the development process seamless, enabling us to focus on creating an intuitive, high-performance platform. With Fused, we could deliver real-time insights and user-friendly visualizations at a scale that was unimaginable only a few years ago.
BlackPrint's platform is transforming real estate decision-making by delivering unprecedented insights to professionals. For example, a retailer in Mexico City utilized our tools to identify a high-traffic site with optimal customer reach, saving weeks of manual analysis and significantly improving accuracy.
Our platform's effectiveness is measured through key metrics like time saved, improved precision, and enhanced ROI for our clients. Users have reported up to a 30% increase in efficiency when planning site expansions or evaluating investments.
Looking ahead, BlackPrint's vision extends beyond Mexico, aiming to revolutionize acquisition intelligence across Latin America. With partners like Fused, we continue to innovate, making geospatial analytics scalable, intuitive, and accessible to professionals at every level.
BlackPrint Technologies is on a mission to redefine real estate intelligence by delivering actionable insights that drive smarter, faster decisions. This journey has been significantly accelerated thanks to Fused, whose serverless data delivery empowered us to process and serve complex geospatial data at an unprecedented scale. Together, we are shaping a future where real estate professionals can access intuitive, data-driven tools that simplify their workflows and enhance their outcomes. Join us in transforming the industry—visit blackprint.ai to see how we are revolutionizing real estate decision-making.
]]>In 2020 while working in Silicon Valley for the county of Santa Clara Valley, I became obsessed with improving monitoring and prevention efforts surrounding Arundo donax. The search and mitigation process this invasive plant species, Arundo donax, was a Sisophisian struggle that had been subject to the same procedures year in and year out, with no progress on beating the spread. To improve the efficacy and efficiency in battling against this notorious weed, I decided to build a tool that would identify the key areas for mitigation - both for the frequency of propagation (occurrence) and for spread potential.
In this blog post I show how I used Fused to create a map of key hotspots where Arundo donax is likely to spread based on built-environment factors derived from Overture Maps data.
To follow along, you check out the UDF associated with the blog post:
In 2020, the problem with managing Arundo was that many agency employees considered this to be a hopeless pursuit, given the exorbitant amount of time cleaning, layering, and calculating the weighted analysis would take. Despite our vast ArcGIS library of tools, engineers, hydrologists, and GIS managers all warned me that I was going to drive myself crazy trying to get the enterprise software to successfully run my analysis. My only regret in building the tool back then was that I didn't have a tool like Fused to expedite the data pulling, processing, and calculating - as it would have saved me from the very lunacy I was warned about.
The obsession with the grass species, Arundo donax, began with an insight to the positive feedback loops created by increased flooding and the spread of invasive species due to climate change. Arundo donax is one of the most invasive plant species worldwide. In addition to destroying biodiversity and disrupting habitats for native species, this large grass also contributes to significant flooding patterns. As weather events become more severe and biodiversity declines, these changes create compounded consequences in our changing climate.
Hot-spot analysis tools using weighted sums can significantly increase accuracy in targeting key areas for prevention. To build a hot-spot analysis for Arundo donax, the following variables need to be scored according to their degree of influence: distance to nutrient loading sources (such as a golf course), distance to a riparian buffer (creek or river), distance to a water-flow disruptor (such as a bridge), and size of the stream.
The key pain points of running a weighted sum on traditional GIS software include:
Using cloud-based systems like Fused can significantly increase calculation speeds, program resilience, and access to public Cloud Native datasets.
After encountering the Fused and learning about how I could improve the speed and geographic spread of site suitability analyses, I wanted to put it to the test by expanding on a previous analysis I did in 2021 using ArcGIS. The original 2021 analysis took several months of data collection, interviews with other local agencies, and extensive data cleaning, standardization, and transformations. I experienced all the aforementioned pain points of hot-spot analysis/weighted sum calculations and more.
Flash forward to today, where I am compiling global data sets, layering them, and deriving statistical insights within 1% of the time that it took me using ArcGIS. Running buffer analyses, weighting variables, and procuring data has taken a fraction of the time for GLOBAL data - and if you remember from before, the previous analysis from 2021 took months for a county-wide calculation and final product.
The most satisfying aspect of my new hot-spot analysis application wasn't just the expansive end product; the process was more seamless and engaging than I had imagined.
I created a UDF with a simple model to identify hotspots susceptible to arundo. The model uses a weighted sum of several base Overture data classes:
The UDF performs the following steps:
Given the complexity of procuring, layering, and interweaving data along with significant wait times and resource consumption, many governmental agencies, non-profits, and even private corporations struggle to run spatial analyses such as site-suitability and hot-spot tools. Insights and tools can be created by improving the speed and efficacy of operations such as weighted sums and fuzzy analysis across spatial layers with UDFs.
Site-suitability and hot-spot analysis go beyond species detection. By simplifying the approach to these types of tools, we can more quickly and accurately detect climate-vulnerable zones, prioritize habitat restoration, and create models to build resilient communities. Industries such as real estate development, retail, and logistics can more quickly understand the variables that affect their businesses by using cloud-based systems like Fused, which can easily manage large datasets.
]]>As communities continue to rebuild and recover from the devastation caused by natural disasters such as wildfires, the question remains: How can we quantify what was lost, especially in the built environment? With the ability to analyze detailed building footprints and overlay fire boundaries, we can begin to answer this by providing a rough estimate of the damage and identifying which structures were impacted by the flames.
In this blog post, Chris Amico shows how by leveraging data such as Overture Building footprints and fire progression maps, we can gain insight into the extent of fire risk. This enables news agencies to derive insights such as count of shops or homes exposed or even assess the capacity of highway routes for residents to evacuate before a prospect fire reaches them.
To follow along, you check out the UDFs and app associated with the blog post:
Fused simplifies the process of spatially joining datasets with Overture Maps data, such as integrating with fire-related datasets.
For this example, we use the Inter Agency Fire Perimeter Historical dataset (published by the National Interagency Fire Center (NIFC)) which includes historical fire perimeters up to 2024. Joining fire perimeters with Overture Buildings and Places data enables us to highlight service gaps or identify regions that may require immediate response.
This demo will first select buildings within a buffer zone to determine which fire perimeter they fall within. Then, it will perform an H3 stats roll-up of business categories from Overture Places, counting the number and types of businesses that fall within each distance range. This will involve rolling up Overture Places business categories into H3 hexagons.
These UDFs return Overture Buildings and Places within a buffer distance from a fire. They offers a simple way to determine the scope of possible fire damage and quickly assess the number of businesses, homes, and other significant structures within the affected area. By adjusting the buffer or selecting fire extent based on dates, users can fine-tune their analysis to gain deeper insights into how far the fire's reach extends and what establishments were most at risk.
First, we create a buffer around the fire perimeter to define the area of interest. This buffer helps us understand the extent of the fire's impact and provides a clear boundary for further analysis.
Next, we load the Overture Buildings dataset and spatially join it with the fire buffer zones. This workflow categorizes buildings based on their proximity to the fire, helping us assess which structures are most at risk.
Finally, we perform a spatial aggregation by calculating the H3 index for the centroids of Overture Places within the fire buffer. This allows us to roll up business categories into H3 hexagons, enabling a holistic overview of business distribution in relation to the fire perimeter.
This kind of analysis helps understand not only the immediate impact but also in planning for future fire preparedness and recovery efforts. In this post, we saw how Overture and Fire-extent data can help us estimate the extent of the damage, from individual buildings to entire neighborhoods.
Organizations looking to integrate these types of perspectives into their workflows could create apps or API services that deliver derivative products, such as GERS lookups to categorize "fire risk" based on buffer proximity.
They could also use Fused HTTPS endpoints from the UDF to return a CSV with the GERS and "Fire Risk" score for buildings within a defined bounding box, as specified by a query parameter. Additionally, they could also use the HTTPS endpoints to automatically enrich the "Fire Risk" column of an arbitrary dataset for any given Building GERS. Users could apply it to any row of their table, with the functionality powered by a "GERS lookup" endpoint using a GERS ID query parameter.
]]>Maribel Hernandez is a computer scientist and researcher at CINVESTAV, a multidisciplinary academic institution in Mexico. She specializes in graph theory in the field of computational genomics and complex networks. In this blog post, Maribel shows how she characterizes cities based on the distribution of businesses by rolling-up business categories by H3.
As someone who works with urban networks, Maribel's focus is on exploring how cities function and how their design impacts inclusivity. The core question driving this analysis is: Does the city have an even distribution of business services? Or are there shortages such as food deserts or unequal access to health facilities?
To follow along, you can clone and run the associated:
Consider these scenarios:
Take, for example, a comparison between 3 key cities in Mexico: Mexico City, León, and Mérida. Are neighborhoods in these cities equally served by essential services such as healthcare, food, and transportation? Could some areas be considered food deserts, while others enjoy easy access to all services?
To answer these questions, I use a method that can scale across cities and ensure consistency. By grouping business categories in Overture Places at the H3 grid level, we can analyze the distribution of services and their accessibility. This approach provides a quantifiable way to compare cities globally, shedding light on the urban inclusivity of different regions. Workflow
The UDF I created follows this workflow:
Preview of UDF on Workbench.
When analyzing the urban fabric of cities like Mexico City, Leon, and Merida, we can obsere distinct patterns of service distribution that manifest as rings around the city center.
Mexico City.
Especially in Leon and Merida, the central area tends to form a cohesive, dense cluster of services. The core zone houses a variety of businesses including health services, retail, and food outlets, which are generally well-connected and easily accessible. Intuitively, the central cluster can be thought of as the "heart" of the city, serving as the primary hub for commerce, social interaction, and access to essential services.
Leon.
Beyond the center, we notice smaller clusters of services scattered across the city, often in the form of "islands." These islands represent pockets of neighborhoods that, while not part of the dense city center, offer an array of unique services. These can serve to highlight the phenomenon of emergent neighborhoods within a greater whole.
Merida.
The periphery of these cities forms another distinct pattern. These outer regions also tend to cluster together in similar service categories, forming a ring that surrounds the central core. It's possible services in these peripheral areas are often more limited in scope and may reflect a focus on residential and less commercial needs, or reflect lower-income neighborhoods with scarcer access to key services.
This ring-like pattern of service distribution suggests a common trend where the core of the city is highly accessible, while the periphery often lacks the same diversity and depth of business services. The islands of services in between can be seen as attempts to bridge this gap, but they are not always as effective in meeting the needs of the population on the periphery.
These spatial patterns offer valuable insights into the inclusivity of urban networks, highlighting areas that may need attention to ensure strategic business placement and guarantee equitable access to essential services.