In Snowflake, a Pipe is the object designed to enable the continuous, near-real-time loading of data from files as soon as they are available in a cloud storage location. Pipes use Snowflake's COPY command to load data and can be associated with a Stage object to monitor for new files. When new data files appear in the stage, the pipe automatically loads the data into the target table.
References:
* Snowflake Documentation on Pipes
* SnowPro Core Certification Study Guide
https://docs.snowflake.com/en/user-guide/data-load-snowpipe-intro.html

To reduce data spilling in Snowflake, using a larger virtual warehouse is effective because it provides more memory and local disk space, which can accommodate larger data operations and minimize the need to spill data to disk or remote storage1. References: [COF-C02] SnowPro Core Certification Exam Study Guide

To monitor updates, such as SCIM API requests sent to Snowflake by the identity provider, an administrator can use the REST EVENT HISTORY feature. This feature allows administrators to query historical data about REST API calls made to Snowflake, including those related to user and role management through SCIM (System for Cross-domain Identity Management).
The REST EVENT HISTORY table function returns information about REST API calls made over a specified period. It is particularly useful for auditing and monitoring purposes, especially when integrating Snowflake with third-party identity providers that use SCIM for automated user provisioning and deprovisioning.
An example query to check for SCIM API requests might look like this:
SELECT * FROM TABLE(information_schema.rest_event_history(date_range_start=>dateadd('hours',-1, current_timestamp()))) WHERE request_type = 'SCIM'; This query returns details on SCIM API requests made in the last hour, including the request type, the identity provider's details, and the outcome of each request.
Reference: Snowflake Documentation on REST EVENT HISTORY (https://docs.snowflake.com/en/sql- reference/functions/rest_event_history.html)

The Access History feature in Snowflake allows administrators to track data access patterns and identify unused data. This information can be used to make decisions about archiving or deleting data to optimize storage and reduce costs.

To optimize query performance on external tables, you can define a materialized view on top of the external table. Snowflake will manage and cache the result for faster retrieval.
Reference:
Snowflake Docs: External Tables with Materialized Views