Ingesting Datasets

Enabling Dataset Ingestion

As described in Exploring Data Sources, users can configure automated ingestion of datasets into the Snowflake database from the Dataset Information view. The Ingestion Tasks tab in the lower half of the view lists every ingestion task for the dataset (including scoped tasks) with its scope, status, and schedule; selecting a task opens its details page, where it can be run, suspended or resumed, deleted, or re-scoped.

To start ingesting a dataset, click Add ingestion task in the Ingestion Tasks tab. This opens a dedicated create page for the dataset, showing:

• Any existing scoped ingestion tasks already configured for the dataset (and, on the Free edition, how many of the per-dataset scoped-task slots are used)
• The Ingestion schedule — on-demand, daily, weekly, or monthly
• Where supported, an optional Limit area or resolution panel to scope the ingestion to a smaller geographic area and/or a coarser resolution — see Scoped Ingestion below

Click Create ingestion task to set up the ingestion task. Setup runs in the background — for larger or scoped datasets it can take a few minutes, and you can safely leave the page while it finishes; the new task already appears in the Ingestion Tasks tab and the Tasks view while it's being set up. On the Free edition, a running count of active ingestion tasks used out of the limit is shown next to the button.

Free edition: The schedule choice offers only On-demand on the Free edition. Daily, Weekly, and Monthly options are not available. To run ingestion again, use the Run Now button in the Tasks view. See Editions for all Free edition limits.

If the data source connection has access credentials configured (see Managing Connections), the ingestion task automatically inherits those settings.

Ingestion Duration

Ingestion runs as an asynchronous task and typically completes within a few minutes to several tens of minutes. The duration depends on two main factors:

1. Data source response times: Many geospatial data sources respond slowly or limit the volume of data returned per request.

2. Dataset size: Even datasets with few rows can contain complex geometry data. Large row counts further increase processing time.

The process is fully automated and requires no intervention once configured.

Note: Ingestion tasks have a maximum duration of 12 hours. If an ingestion run does not complete within this time, it is automatically stopped and the task is marked as failed. This limit accommodates even the largest datasets and slowest data sources. If a task consistently fails due to timeouts, contact support via the Geo Data Connector listing on Snowflake Marketplace.

Scoped Ingestion

For datasets that are too large to ingest in full — or that exceed the Free edition's row limit — Geo Data Connector supports scoped ingestion: limiting a subscription to a smaller geographic area and/or a coarser resolution, instead of ingesting the entire dataset. Scoped ingestion is currently available for coverage-type (WCS, OGC API Coverages), image-type (WMS, OGC API Maps), tile (OGC API Tiles vector tiles; WMTS and OGC API Tiles map tiles on geographic latitude/longitude tile grids), and vector-type (WFS, OGC API Features) data sources; other data source types report that scoped ingestion is not yet available for them, with support for further types planned.

A scope narrows ingestion in two ways:

• Area — ingest only the portion of the dataset within a chosen geographic extent
• Resolution — ingest at a coarser resolution than the source's native resolution, where the data source type supports it

Tiled map sources: the resolution lever is the zoom level — tile-grid datasets ingest at the source's published zoom levels. You can keep the full area at a coarser zoom level, or keep full detail on a smaller area; if the size check rejects your selection, choose a coarser zoom level or a smaller area.

To create a scoped ingestion task, click Add ingestion task in the dataset's Ingestion Tasks tab (see Enabling Dataset Ingestion above), then use the Limit area or resolution panel on the create page: draw or enter a geographic area on the map, and, where the data source type supports it, choose a coarser resolution. Click Create ingestion task to submit — the Connector checks the selected scope's estimated size and rejects the task with an explanatory message if the area or resolution cannot be scoped as selected.

Once created, a scoped ingestion task is immutable — its area and resolution cannot be changed afterward. To adjust the scope, create a new task with the desired area and/or resolution (see Re-scoping below); data already ingested by the original task is preserved.

If a scoped ingestion task's selected area turns out to contain no data (for example, it covers only cells the dataset has no values for), the run completes as skipped with an explanatory reason — not as a failure — and the ingestion task stays enabled. To get data, create a new task with a different area, or ingest the full dataset.

Needs re-scope: A scoped ingestion task can outgrow its size budget over time — for example, if the source dataset grows, or if a Free edition task's tier changes. When this happens, the Tasks view and Task Details view show a Needs re-scope badge. The data already ingested remains available; re-scope the task (see below) to continue receiving updates.

Re-scoping

From the Task Details view of a task showing the Needs re-scope badge (or any scoped task you want to adjust), click Re-scope. This opens the create page — the same page used to add a new ingestion task — with the area and resolution prefilled from the task being re-scoped. Adjust the area and/or resolution as needed and click Create ingestion task: a new ingestion task is created with the new scope, and the previous task is automatically suspended. Data already ingested by the previous task is preserved.

Monitoring Ingestion Tasks

Select Tasks from the main menu to open the Ingestion Tasks view, where you can monitor and manage all configured ingestion tasks.

Ingestion Tasks view showing all configured tasks with status and schedule

Task Details

Column	Description
Task Name	A unique technical identifier of the ingestion task
Type	The type of the data source (e.g., WFS)
Health	A colored indicator showing the current connectivity status of the data source used by this task. See Managing Connections for status meanings.
Data Source	The title of the geospatial data source. Falls back to the source URL for tasks created before this feature.
Dataset	The title of the dataset being ingested. Falls back to the technical layer name for older tasks.
Target Table	The fully qualified name of the target table in Snowflake, following the format <database>.<schema>.<table> — based on the ingestion target database configured in the application settings
Status	The status of the most recent ingestion run. While ingestion is running, shows the current phase (e.g., "IN PROGRESS - 3/6 Retrieving"). When failed, a tooltip icon shows the reason for the failure. Possible status values: N/A, IN PROGRESS, SUCCESS, FAILED, SKIPPED (the run was skipped — hover the information icon for the reason; see Ingestion Task Statuses below), or, for a task that was just created, Setting up…, Setup failed, or Not created (see Ingestion Task Statuses below).
Next Run	The scheduled start time of the next ingestion run (GMT)
Last Run	The date and time of the most recent ingestion run (an ingestion that could not be started does not change it)
Created	The date and time when the ingestion task was created (GMT)
View dataset	Opens the dataset's detail page (the Connector looks the dataset up at the source — if the source is temporarily unavailable or the dataset was removed, a notice explains why the page could not be opened). From there you can review the data or add another ingestion task for it, including a scoped one.
On	Whether the task is active (ON) or suspended (OFF). Click the toggle to switch between states. For a failed, suspended task the control reads Retry — clicking it re-enables the task (a scheduled task then runs at its next scheduled time; for an on-demand task, use Run Now to retry the ingestion).
Schedule	The update frequency selected for the task (On-demand, Daily, Weekly, or Monthly). On-demand tasks have no recurring schedule and show N/A for Next Run.
Actions	Immediate ingestion triggering and deletion of the ingestion task available via action icons

Actions

Action	Description
Immediate ingestion	Triggers ingestion immediately, regardless of the configured schedule. The status changes to IN PROGRESS until complete. When the request is accepted the Connector confirms it with "Ingestion started."; if it cannot be started you see a message that says why (for example, the Free edition daily limit) or that you can try again in a moment (if the data service cannot start the run, the run history shows it as FAILED after you Refresh, with the reason), and the message stays until you close it.
Delete	Removes the ingestion task

Task Details and Run History

Click any task row in the Ingestion Tasks view to open the Task Details view. This view shows the task's metadata cards at the top and a full history of ingestion runs below.

Task Details view with metadata cards and ingestion run history

The metadata cards summarize the task configuration and current state:

Card	Description
Source Type	Data source type (e.g., WFS, WMS)
Data Source	Title of the data source
Dataset	Title of the dataset being ingested
Target Table	Fully qualified Snowflake target table
Status	Current ingestion status with color-coded indicator
Next Ingestion	Estimated completion time for the next or current run
Last Ingestion	Completion time of the most recent run
Created	When the ingestion task was created
State	Whether the task is Enabled or Disabled
Schedule	Update frequency (On-demand, Daily, Weekly, or Monthly). On-demand tasks show N/A for Next Ingestion.

The Ingestion Task Runs table below shows the history of all ingestion runs for the task:

Column	Description
Name	Run identifier
Start Time	When the run started
End Time	When the run completed (N/A if still running)
Status	Run outcome. While running, shows the current phase (e.g., "RUNNING - 3/6 Retrieving"). When failed, a tooltip icon shows the error reason. Possible values: PENDING (the run was just started and is waiting for its first update), RUNNING, SUCCEEDED, FAILED, SKIPPED (the run completed without ingesting data — because the Free edition daily limit was reached, resetting at midnight UTC; because a scoped ingestion task's selected area contained no data; or because the Connector received a response it could not use while preparing to load the run's data, in which case the task stays enabled and tries again at its next run, and an on-demand task can simply be run again later; a skipped run still counts toward the Free edition daily limit; hover the information icon for the specific reason), POSTPONED (the data source was temporarily unavailable when the run started; scheduled tasks retry automatically — the Connector determines the interval based on data-source conditions — on-demand tasks return to inactive and can be run again manually), CANCELLED (the task was suspended while this run was in progress — resume the task, and for an on-demand task run it again, to resume tracking; the run then completes normally).
Duration	How long the run took
Trigger	What started the run (scheduled or manual)
Rows Loaded	Number of rows written to the target table

Note: The run history status values (RUNNING, SUCCEEDED, FAILED) reflect the outcome of each individual ingestion run. These are distinct from the task-level statuses (such as IN PROGRESS, SUCCESS, RETRYING, FAILED) shown in the task list, which reflect the overall state of the task.

Run history shows the most recent 90 days of runs (up to the last 100); older records are cleaned up automatically.

Click Refresh to update the run history with the latest data. A run you just started shows as PENDING - Starting until its first status update.

Suspending a task during an active run: the run's row is marked CANCELLED and processing pauses from the Connector's point of view. Resume the task — and for an on-demand task, run it again — to resume tracking; the run then completes normally and delivers its data.

Target Table Naming

Each ingested dataset is written to a fully qualified target table in Snowflake, identified by the format:

<database>.<schema>.<table>

Component	Description
Database	This is the Connector-wide ingestion target database, configured in the Application Settings. All ingested data is stored within this database.
Schema	The schema is automatically generated based on the domain of the data source URL (e.g., example.org → org_example). Each unique domain results in a separate schema, organizing datasets by their source.
Table	The table name is derived from the data source type (e.g., WFS) and the dataset name. To ensure uniqueness—especially when multiple datasets have similar names or come from the same source—a randomized suffix may be appended to the table name, depending on the naming context and source type.

Ingestion Task Statuses

Status	Description
IN PROGRESS	Ingestion is currently underway. The status column shows the current execution phase (e.g., "IN PROGRESS - 3/6 Retrieving") so you can track progress through the six phases: Triggering, Retrieving, Processing, Loading, Transforming, and Finalizing.
SUCCESS	The most recent ingestion completed successfully. Data from the source was ingested into the target Snowflake table without errors.
RETRYING	The most recent ingestion attempt failed, but the Connector is automatically retrying. Depending on the failure, the Connector may first retry within minutes (shown as "Retrying shortly"); scheduled tasks are then retried on the following days (shown as "Retry 1/2", "Retry 2/2") before being marked as FAILED, while on-demand tasks are marked as FAILED after any retries within minutes.
INACTIVE	The ingestion task exists but has been disabled. If the Connector switched the task off because it is no longer set up, an information icon explains this; select Resume to try setting it up again, and if that does not work, delete the task and create it again.
FAILED	The most recent ingestion attempt did not complete successfully. Hover over the information icon next to the status to see the failure reason, including which phase the error occurred in (e.g., "Failed in phase 2/6 Retrieving: Data retrieval took too long and was stopped.").
SKIPPED	The run was skipped without ingesting data. Hover over the information icon next to the status to see the reason. Three cases: the Free edition daily ingestion limit (10 runs per day) was reached — the counter resets at midnight UTC and ingestion resumes with the next scheduled or manual run; a scoped ingestion task's selected geographic area contained no data — try a different area, or create a task for the full dataset; or the Connector received a response it could not use while preparing to load the run's data — nothing was changed, the task stays enabled and tries again at its next run (an on-demand task can simply be run again later), and if the problem persists, contact support. A skipped run still counts toward the Free edition daily limit.
Setting up…	The ingestion task was just created and is still being set up in the background. This is temporary — the task moves to another status once setup finishes. You can safely navigate away; setup continues regardless.
Setup failed	The ingestion task could not be fully set up. Hover over the information icon next to the status to see why. Delete the task and try creating it again.
Not created	The requested ingestion task could not be created as configured (for example, the selected area or resolution could not be applied). Hover over the information icon next to the status to see why, then create a new task with an adjusted area or resolution.

Failed Task Handling

Failed tasks are automatically disabled to prevent repeated unsuccessful attempts. To resume, use Retry on the task's row in the Tasks view, or on its Task Details page.

Tasks include built-in retry logic — a task is retried multiple times before being marked as failed. Failures are often caused by temporary outages of the data source service, so re-enabling a failed task usually succeeds on the next run.

FAILED at creation vs. FAILED after retries: if an ingestion task shows FAILED immediately after you create it — with an empty run history — the task setup itself could not be completed and no ingestion run was ever attempted; hover over the information icon next to the status for the reason (for example, the Free edition's ingestion task limit, or a temporary issue reaching the data service). This is different from a task that shows FAILED after one or more runs in its history — that task did attempt ingestion and failed after exhausting its retries (see "Understanding Failure Patterns" below). In both cases, re-enable the task once the underlying issue is resolved.

Tip: Enable Task Failure email notifications in Application Settings to receive an alert when an ingestion run fails or will be retried, when a run is skipped because the Connector could not load its data, or when the Connector switches a task off because it is no longer set up. This way you can respond promptly without needing to check the Tasks view manually.

In some cases, a data source may advertise datasets that cannot actually be retrieved — for example, when the service lists a dataset but the server rejects the data request, requires non-standard query parameters, or has a server-side misconfiguration. The Connector validates data accessibility during ingestion and marks the task as failed if the data cannot be downloaded, rather than leaving the task in a perpetual processing state.

Understanding Failure Patterns

The Task Details view provides run history that can help determine whether a failure is temporary or persistent:

• A task that previously succeeded but now fails is typically experiencing a temporary data source issue — re-enabling it is usually sufficient.
• A task that has never succeeded may indicate a persistent problem such as a dataset size limit, incompatible data format, or non-standard service behavior.
• The Rows Loaded column in the run history indicates whether any data was retrieved before the failure occurred — a value of zero suggests the failure happened early (connectivity, permissions, or size limit), while a non-zero value suggests a mid-transfer issue.

For detailed troubleshooting guidance, see Troubleshooting: Ingestion Task Failed.

Managing Ingestion Tasks

Ingestion tasks are managed from the Tasks view (all tasks) and the dataset's Ingestion Tasks tab (tasks for one dataset) — both list tasks and let you click a row to open its Task Details page. Run, suspend/resume/retry, re-scope, and delete actions are available from the task's row in either list, and from the Task Details page itself; a dataset no longer has its own inline enable toggle — all task management happens through the task's own row or details page.

Suspend an Ingestion Task

Switch the On toggle to the off position on the task's row in the Tasks view, or use the Suspend button on the task's details page.

When the task is suspended the Connector confirms it with "Ingestion task updated." If the task was suspended but the Connector could not confirm the change with the data service, a warning says so and suggests suspending again in a moment. If the task could not be suspended, an error message appears, stays until you close it, and the toggle returns to its previous position.

Note: Suspending the task stops further updates to the dataset but does not remove any data already ingested into the Snowflake target table if the task had completed successfully before suspending.

Resume or Retry an Ingestion Task

Switch the On toggle to the on position on the task's row in the Tasks view, or use the button on the task's details page — labeled Resume for a suspended task, or Retry for a task that is disabled because its most recent run failed. Resuming a scheduled task does not start an immediate run — the next ingestion occurs at the schedule's next regular time, shown in the Next Run column; for an on-demand task, or to retry immediately, use Run Now.

The Connector confirms a successful change with "Ingestion task updated." If the task could not be resumed, an error message appears, stays until you close it, and the toggle returns to its previous position. A task that has no schedule cannot be resumed; the message says so ("This ingestion task has no schedule, so it cannot be resumed.").

Delete an Ingestion Task

Click the delete icon in the Actions column of the corresponding task's row on the Tasks view, or use the Delete button on the task's details page.

When the task is deleted the Connector confirms it with "Ingestion task deleted."; on the task's details page you are then returned to the Tasks view. If the task could not be deleted — for example because the data service did not accept the removal — an error message appears, stays until you close it, and you remain where you were. The task is still listed, and it is suspended for now; delete it again in a moment, or resume it if you want to keep it. If the message says the ingestion task is still finishing its current run, nothing was deleted; wait a moment and delete it again.

Note: Deleting the task stops further updates, permanently removes the ingestion task, but does not delete any data already ingested into the Snowflake target table if the task had completed successfully before deletion.

Choosing an Update Schedule

The update frequency (labeled Ingestion schedule on the create page) is chosen when the ingestion task is created. To change how often a dataset updates, create a new ingestion task with the desired schedule (and delete or suspend the old one). A newly scheduled task does not start an immediate run — the next ingestion occurs at the schedule's next regular time. Available options are:

• On-demand (default) — Runs once immediately when created. Re-run at any time using the Run Now button. No recurring schedule; Next Run always shows N/A.
• Daily — Runs once per day.
• Weekly — Runs once per week.
• Monthly — Runs once per month.

Free edition: Only On-demand is available on the Free edition. The Daily, Weekly, and Monthly options are not shown.

Data Source Type Differences

While the ingestion workflow is consistent across all data source types, the output format varies depending on the type of data being ingested. Each traditional OGC service type has a modern OGC API equivalent that produces the same output format.

Vector Data Sources (WFS, OGC API Features)

WFS (Web Feature Service) and OGC API Features provide vector data—features with geometry and attributes. When ingested:

• Output format: GeoJSON stored in a Snowflake table
• Schema: Columns match the feature attributes from the source
• Geometry: Stored as GeoJSON in a dedicated geometry column
• Querying: Use standard SQL with Snowflake's geospatial functions (see Working with Ingested Data)

Raster/Image Data Sources (WMS, WMTS, WCS, OGC API Maps, OGC API Coverages)

WMS, WMTS, WCS, and their OGC API equivalents (OGC API Maps, OGC API Coverages) provide raster data—map images or coverage data. When ingested, two artifacts are always created:

1. Pixel data table (primary target table): Contains per-pixel data with spatial coordinates, band values, and GeoJSON geometry. This is the table shown in the Ingestion Target Table column on the Tasks view.
2. Metadata table: A companion table named with the __metadata suffix (e.g., WMS__MY_LAYER_A1B2C3D4__metadata) containing image properties such as dimensions, coordinate reference system, band information, and geographic extent.

WMS, WMTS, and their OGC API equivalents (OGC API Maps) also produce a third artifact:

3. Image stage: A Snowflake stage containing the georeferenced image file(s) for direct download.

WCS and OGC API Coverages deliver raw raster data (elevation, temperature, satellite bands) and do not produce an image stage.

For details on querying these tables and accessing image files, see Working with Ingested Data.

WMS (Web Map Service)

WMS delivers rendered map images. Key characteristics:

• Layer selection: Choose specific map layers to ingest from the dataset list
• Image format: The Connector requests the best available format (GeoTIFF preferred, then PNG)
• Coordinate system: Images are georeferenced using the layer's native coordinate reference system
• Use case: Ideal for basemaps, thematic maps, and visualizations where you need the rendered appearance

WMTS (Web Map Tile Service)

WMTS provides pre-rendered map tiles. Key characteristics:

• Tiled delivery: Maps are delivered as a grid of tiles at multiple zoom levels
• Efficient access: Tiles are optimized for fast web display
• Merged output: Tiles are merged into a single image during ingestion
• Use case: Ideal for basemaps and services designed for web mapping applications

WCS (Web Coverage Service)

WCS provides raw raster coverage data. Key characteristics:

• Raw data: Delivers actual raster values (e.g., elevation, temperature, satellite bands)
• Multi-band support: Can include multiple data bands per coverage
• Large dataset handling: Datasets exceeding 500 MB are automatically split into smaller parts for reliable processing
• Maximum dataset size: Datasets exceeding the current maximum size limit cannot be ingested. If a coverage exceeds this limit, the ingestion task will fail immediately. This limit is expected to be raised in a future release. Contact support for assistance with bbox subsetting to ingest a specific geographic region of the coverage.
• Use case: Ideal for scientific analysis, remote sensing data, and raster analytics

Note: Large WCS and OGC API Coverages datasets may take longer to ingest due to automatic tiling and processing. The ingestion duration shown in the Task Details view accounts for this additional processing time.

OGC API Equivalents

The modern OGC API standards provide the same data types as their traditional counterparts, and produce identical output when ingested:

OGC API Service	Traditional Equivalent	Output Type
OGC API Features	WFS	Vector (GeoJSON table)
OGC API Maps	WMS	Raster (pixel table + metadata + image)
OGC API Tiles	WMTS	Map tiles: Raster (pixel table + metadata + image); vector tiles: Vector (GeoJSON table)
OGC API Coverages	WCS	Raster (pixel table + metadata)

The ingestion workflow, target table format, and all features described above apply equally to both traditional and OGC API data sources.