1. insupdel4stac package

INSUPDEL4STAC

A python-based module to ingest STAC metadata catalogs into STAC-databases like pgSTAC

class insupdel4stac.Connector(service: str = 'pgstac', pgstac_properties: dict = {}, stacapi_properties: dict = {})

Bases: object

This class provides access to the loader and PgstacDB. Conversely, it also affords the opportunity to configure the connection to the database and API. The setup can be achieved through the utilization of either environment variables and constant variables, or by utilizing pgstac_properties or stacapi_properties.

Parameters:

• service (str, optional) – Service to be configured. Defaults to “pgstac”.

• pgstac_properties (dict, optional) – Properties for pgstac. Defaults to dict().

• stacapi_properties (dict, optional) – Properties for stacapi. Defaults to dict().

pgstac_connection()

pgstac_properties: dict

To learn more about constructing the dictionary, see pgstac_properties.

properties_config(service: str, properties: dict = {}) → dict

service: str

For more information see service_type.

stacapi_connection(properties: dict = {})

stacapi_properties: dict

To learn more about constructing the dictionary, see stacapi_properties.

class insupdel4stac.Deleter(collections_list: list, service_type: str, table_details: list | None = None, loader: Loader | None = None, pgstacdb: PgstacDB | None = None, pgstac_properties: dict | None = None, stacapi_properties: dict | None = None, logger_properties: dict | None = None)

Bases: object

This is a class for deleting items and collections from pgSTAC and STAC-API.

Parameters:

• collections_list (list) – A list of collections to be deleted.

• service_type (str) – The service type to be used for deleting. It can be pgstac or stacapi.

• table_details (list, optional) – A list of dictionaries that contains the details of the tables. Defaults to None.

• loader (Loader, optional) – The loader object to be used for deleting.

• pgstacdb (PgstacDB, optional) – The PgstacDB object to be used for deleting.

• pgstac_properties (dict, optional) – The properties of pgstac. Defaults to None.

• stacapi_properties (dict, optional) – The properties of stacapi. Defaults to None.

• logger_properties (dict, optional) – The properties of logger. Defaults to None.

delete(collection: Collection, service: str, loader: Loader | None = None, table_details: list | None = None)

This function ingests the items and collections from pgSTAC and STAC-API according to the service_type and table_details.

pgstac_collection_delete(collection: Collection, PgstacDB: PgstacDB)

This function deletes the collection from pgSTAC.

pgstac_item_delete(item: Item, collection: Collection, PgstacDB: PgstacDB)

This function deletes the item from pgSTAC.

stacapi_collection_delete(collection: Collection)

This function deletes the collection from STAC-API via DELETE method.

stacapi_item_delete(item: Item, collection: Collection)

This function deletes the item from STAC-API via DELETE method.

class insupdel4stac.ExistenceValidator(stac_dir: str = '/home/docs/checkouts/readthedocs.org/user_builds/insupdel4stac/checkouts/stable/docs', default_catalog_name: str = 'catalog.json', collection_id: str | None = None)

Bases: object

A class for verifying the main STAC catalog’s existence. This class is implemented in InsUpDel4STAC.

Parameters:

• stac_dir (str) – Directory of the main STAC catalog (*)

• default_catalog_name (str, optional) – Name of the main STAC catalog. default is “catalog.json”.

• logger_properties (dict, optional) – A dictionary of properties for logger. default is None.

catalog_existance()

This function verifies the existence of the main STAC catalog.

default_catalog_name: str

Name of the main STAC catalog. default is “catalog.json”.

logger_properties: dict | None

A dictionary of properties for logger. default is None. You can look at keys in Logger class.

stac_dir: str

Directory of the main STAC catalog. It can be a relative or absolute path.

class insupdel4stac.InsUpDel4STAC(stac_dir: str = '/home/docs/checkouts/readthedocs.org/user_builds/insupdel4stac/checkouts/stable/docs', action: str = 'Insert', table_details: list | None = None, default_catalog_name: str = 'catalog.json', service_type: str | None = 'pgstac', stacapi_properties: dict = {}, pgstac_properties: dict = {}, logger_properties: dict = {})

Bases: object

The current class serves as the primary implementation for performing Insert, Update, and Delete actions on STAC-Catalogs. The class gets the STAC directory and the action type, for initiating the subsequent process.

Parameters:

• stac_dir (str, optional) – The directory where the STAC Catalog is located. The default value is set to os.getcwd().

• action (string, optional) – The type of action. The selection will be made from the options of Insert, Update, and Delete. The default setting is set to Insert.

• table_details (Union[list, None], optional) – The permissible objects for input actions are a list of dictionaries, STAC-Collections, and STAC-Items. For further information about how to construct this argument, please refer to table_details. The default value is set to None.

• default_catalog_name (str, optional) – It represents the default name of the STAC-catalog. The default value is set to “catalog.json`.

• service_type (Union[str, None], optional) – It represents the type of service. The selection will be made from the pgstac and stacapi components. For further information on this argument, please refer to service_type. The default value is set to pgstac.

• stacapi_properties (dict, optional) – The current dictionary provides definitions for STAC-API environments. To obtain additional information on the construction of stacapi_properties, please refer to stacapi_properties. The default value is set to an empty dictionary, dict().

• pgstac_properties (dict, optional) – The current dictionary provides definitions for pgSTAC environments. To obtain additional information on the construction of pgstac_properties, please refer to pgstac_properties. The default value is set to an empty dictionary, dict().

• logger_properties (dict, optional) – The current dictionary provides definitions for logger properties. To obtain additional information on the construction of logger_properties, please refer to logger_properties. The default value is set to an empty dictionary, dict().

action: str

The variable at present denotes the category of action. The decision will be taken from the alternatives of Insert, Update, and Delete.

default_catalog_name: str

The variable now denotes the default JSON name for the STAC-catalog.

dispatchCollectionActions(action: str, collections_list: list, service_type: str, table_details: list | None = None, loader: Loader | None = None, pgstacdb: PgstacDB | None = None, pgstac_properties: dict = {}, stacapi_properties: dict = {}, logger_properties: dict = {})

The provided function serves as a dispatcher, responsible for directing actions to their respective classes: Deleter, Ingester, and Updater.

Parameters:

• action (str) – The type of action. The selection will be made from Insert, Update, and Delete. The default setting is set to Insert.

• collections_list (list) – The permissible pySTAC Collection objects for input actions.

• service_type (str) – It represents the type of service.

• table_details (Union[list, None], optional) – The permissible objects for input actions

• dictionaries (are a list of) –

• STAC-Collections –

• information (and STAC-Items. For further) –

• argument (about how to construct this) –

• to (please refer) –

:param table_details. The default value is: :param set to None.: :param loader: The loader object connector for pgSTAC. The

default value is set to None.

Parameters:

• pgstacdb (Union[object, None], optional) – The pgstacdb object connector for pgSTAC. The default value is set to None.

• pgstac_properties (dict, optional) – An environment dictionary for pgSTAC. To obtain additional information on the construction of pgstac_properties, please refer to pgstac_properties. The default value is set to an empty dictionary, dict().

• stacapi_properties (dict, optional) – An environment dictionary for STAC-API. To obtain additional information on the construction of stacapi_properties, please refer to stacapi_properties. The default value is set to an empty dictionary, dict().

• logger_properties (dict, optional) – logger properties. logger_properties.

logger_properties: dict

A dictionary of properties for logger. default is None. You can look at keys in Logger class.

pgstac_properties: dict

The existing variable denotes the dictionary that furnishes definitions for STAC-API environments. The dictionary contains the following keys:

POSTGRES_HOST_READER (str): The host name of the database server for reading.

POSTGRES_HOST_WRITER (str): The host name of the database server for writing.

POSTGRES_PORT (str): The port number of the database server.

POSTGRES_USER (str): The username for the database server.

POSTGRES_PASSWORD (str): The password for the database server.

POSTGRES_DB (str): The name of the database.

PGUSER (str): The username for the database server.

PGPASSWORD (str): The password for the database server.

PGHOST (str): The host name of the database server.

PGDATABASE (str): The name of the database.

PgstacDB_dsn (str): The data source name (DSN) for the database connection.

PgstacDB_pool (object): The connection pool for the database.

PgstacDB_connection (object): The connection object for the database.

PgstacDB_commit_on_exit (bool): The boolean value that determines whether the

PgstacDB_debug (bool): The boolean value that determines whether the debug mode is enabled.

PgstacDB_use_queue (bool): The boolean value that determines whether the queue is used.

service_type: str | None

The variable now denotes the category of service. The pick will be made from the “pgstac” and “stacapi” components.

pgstac (str): The execution of this operation is carried out on the pgSTAC

service using SQL-based commands by the pypgSTAC framework.

stacapi (str): The execution of this procedure is carried out within the STAC-API

by utilizing POST, PUT, and DELETE requests to conduct the operations of inserting, updating, and deleting the STAC-Collections and STAC-Items, respectively. It is crucial to recognize that the implementation of the mentioned methods relies on the authorization and accessibility of the STAC-API services, together with the existence of authentication protocols to enable the execution of the designated tasks. To obtain additional details regarding the authentication procedure, please see the documentation for stacapi_properties.

stac_dir: str

The variable at hand denotes the specific directory in which the STAC Catalog is presently situated.

stacapi_properties: dict

The existing variable denotes the dictionary that furnishes definitions for STAC-API environments. The dictionary contains the following keys:

stacapi_url (str): The URL of the STAC-API service.

auth (dict, optional): The dictionary that contains the username and password

for the STAC-API service. e.g. {“username”: “username”, “password”: “password”}

timeout (int): The timeout value for the STAC-API service.

verify (bool): The boolean value that determines whether the verification is enabled.

table_details: list | None

The current variable represents the allowable entities for input actions as a list of dictionaries, STAC-Collections, and STAC-Items. The structure of the dictionary list can be exemplified as follows:

class insupdel4stac.Inserter(collections_list: list, service_type: str, table_details: list | None = None, loader: Loader | None = None, pgstacdb: PgstacDB | None = None, pgstac_properties: dict | None = None, stacapi_properties: dict | None = None, logger_properties: dict | None = None)

Bases: object

This is a class for ingesting items and collections into pgSTAC and STAC-API.

Parameters:

• collections_list (list) – A list of collections to be ingested.

• service_type (str) – The service type to be used for ingestion. It can be pgstac or stacapi.

• table_details (list, optional) – A list of dictionaries that contains the details of the tables. Defaults to None.

• loader (Loader, optional) – The loader object to be used for ingestion.

• pgstacdb (PgstacDB, optional) – The PgstacDB object to be used for deleting.

• pgstac_properties (dict, optional) – The properties of pgstac. Defaults to None.

• stacapi_properties (dict, optional) – The properties of stacapi. Defaults to None.

• logger_properties (dict, optional) – The properties of logger. Defaults to None.

ingester(collection: Collection, service: str, loader: Loader | None = None, table_details: list | None = None)

This function ingests the items and collections into pgSTAC and STAC-API according to the service_type and table_details.

pgstac_collection_ingest(collection_self_path: str, loader: Loader)

This function ingests the collections into pgSTAC.

pgstac_item_ingest(item: Item, PgstacDB: PgstacDB)

This function ingests the items into pgSTAC.

stacapi_collection_ingest(collection: Collection)

This function ingests the collections into STAC-API via POST method.

stacapi_item_ingest(item: Item)

This function ingests the items into STAC-API via POST method.

class insupdel4stac.Logger(logger_properties: dict[str, Any] | None = {})

Bases: object

A class-based logger for INSUPDEL4STAC. It supports all the handlers from the standard python logging library.

Parameters:

logger_properties (dict, optional) –

Logger properties. Defaults to dict(). It’s optional and has the following keys:

logger_msg (str, optional)

logger_handler (str, optional)

logger_name (str, optional)

logger_id (str, optional)

logger_level (str, optional)

logger_formatter (str, optional)

logger_handler_host (str, optional)

logger_handler_port (str, optional)

logger_handler_url (str, optional)

logger_handler_method (str, optional)

logger_handler_secure (bool, optional)

logger_handler_credentials (tuple, optional)

logger_handler_context (tuple, optional)

logger_handler_filename (str, optional)

logger_handler_mode (str, optional)

logger_handler_encoding (str, optional)

logger_handler_delay (bool, optional)

logger_handler_errors (str, optional)

logger_handler_mailhost (str, optional)

logger_handler_fromaddr (str, optional)

logger_handler_toaddrs (str, optional)

logger_handler_subject (str, optional)

logger_handler_timeout (str, optional)

Null_Handler()

This is a function to return a NullHandler

logger_properties: dict[str, Any] | None

A dictionary that contains all the logger properties.

It is optional and it is set to None by default. The following keys are supported:

logger_msg (str, optional):

Logger message. Defaults to None. But it is required when you want to log a message.

logger_handler (str, optional):

Logger handler. Defaults to NullHandler. Check the following website for more information:

https://docs.python.org/3/library/logging.handlers.html#module-logging.handlers

logger_name (str, optional):

Logger name. Defaults to INSUPDEL4STAC. It’s required when you choose HTTPHandler as logger_handler.

logger_id (str, optional):

Logger id. Defaults to 1. It’s required when you choose HTTPHandler as logger_handler.

logger_level (str, optional):

Logger level. Defaults to DEBUG. It’s optional. For more information check the following website:

https://docs.python.org/3/library/logging.html#levels

logger_formatter (str, optional):

Logger format. Defaults to %(levelname)-8s %(asctime)s t %(filename)s @function %(funcName)s line %(lineno)s - %(message)s. For more information check the following website:

https://docs.python.org/3/library/logging.html#formatter-objects

logger_handler_host (str, optional):

Logger host. Sets the value to ‘None’ by default. It is required when HTTPHandler or SocketHandler are selected as the logger_handler. The logger_handler will be set to ‘NullHandler’ if HTTPHandler or SocketHandler is selected as the logger_handler value and neither logger_handler_host nor logger_handler_port nor are specified.

logger_handler_port (str, optional):

Logger port. Sets the value to ‘None’ by default. It is required when HTTPHandler or SocketHandler are selected as the logger_handler. The logger_handler will be set to ‘NullHandler’ if HTTPHandler or SocketHandler is selected as the logger_handler value and neither logger_handler_host nor logger_handler_port are specified.

logger_handler_url (str, optional):

Logger url. Sets the value to ‘None’ by default. It is required when HTTPHandler is selected as the logger_handler. The logger_handler will be set to ‘NullHandler’ if HTTPHandler is selected as the logger_handler value and neither logger_handler_url is specified.

logger_handler_method (str, optional):

HTTP methods. It supports sending logging messages to a web server, using either GET or POST semantics. Sets the value to ‘None’ by default. It is required when HTTPHandler is selected as the logger_handler. The logger_handler will be set to ‘NullHandler’ if HTTPHandler is selected as the logger_handler value and logger_handler_method is not specified.

logger_handler_secure (bool, optional):

HTTP secure. Sets the value to ‘False’ by default. It is utilized when HTTPHandler or SMTPHandler are selected as the logger_handler. But it is optional in both logger handlers.

logger_handler_credentials (tuple, optional):

HTTP credentials. Sets the value to ‘None’ by default. It is utilized when HTTPHandler or SMTPHandler are selected as the logger_handler. But it is optional in both logger handlers.

logger_handler_context (tuple, optional):

HTTP context. Sets the value to ‘None’ by default. It is utilized when HTTPHandler is selected as the logger_handler. But it is optional in both logger handlers.

logger_handler_filename (str, optional):

File name. Sets the value to ‘None’ by default. It is required when FileHandler or WatchedFileHandler are selected as the logger_handler. The logger_handler will be set to ‘NullHandler’ if FileHandler or WatchedFileHandler is selected as the logger_handler value and logger_handler_filename is not specified.

logger_handler_mode (str, optional):

File mode. Sets the value to ‘None’ by default. It is required when FileHandler or WatchedFileHandler are selected as the logger_handler. The logger_handler will be set to ‘NullHandler’ if FileHandler or WatchedFileHandler is selected as the logger_handler value and logger_handler_mode is not specified.

logger_handler_encoding (str, optional):

File encoding. Sets the value to ‘None’ by default. It is utilized when FileHandler or WatchedFileHandler are selected as the logger_handler. But it is optional in both logger handlers.

logger_handler_delay (bool, optional):

File delay. Sets the value to ‘False’ by default. It is utilized when FileHandler or WatchedFileHandler are selected as the logger_handler. But it is optional in both logger handlers.

logger_handler_errors (str, optional):

File errors. Sets the value to ‘None’ by default. It is utilized when FileHandler or WatchedFileHandler are selected as the logger_handler. But it is optional in both logger handlers.

logger_handler_mailhost (str, optional):

Mail host. Sets the value to ‘None’ by default. It is required when SMTPHandler is selected as the logger_handler. The logger_handler will be set to ‘NullHandler’ if SMTPHandler is selected as the logger_handler value and logger_handler_mailhost is not specified.

logger_handler_fromaddr (str, optional):

Mail from address. Sets the value to ‘None’ by default. It is required when SMTPHandler is selected as the logger_handler. The logger_handler will be set to ‘NullHandler’ if SMTPHandler is selected as the logger_handler value and logger_handler_fromaddr is not specified.

logger_handler_toaddrs (str, optional):

Mail to address. Sets the value to ‘None’ by default. It is required when SMTPHandler is selected as the logger_handler. The logger_handler will be set to ‘NullHandler’ if SMTPHandler is selected as the logger_handler value and logger_handler_toaddrs is not specified.

logger_handler_subject (str, optional):

Mail subject. Sets the value to ‘None’ by default. It is utilized when SMTPHandler is selected as the logger_handler. But it is optional in both logger handlers.

logger_handler_timeout (str, optional):

Mail timeout. Sets the value to ‘None’ by default. It is utilized when SMTPHandler is selected as the logger_handler. But it is optional in both logger handlers.

class insupdel4stac.TableDetails(table_details: list | None = None, logger_properties: dict = {})

Bases: object

This class provides access to the table details. It is used to determine which tables are to be updated, inserted, or deleted.

Parameters:

table_details (list) – List of dictionaries containing the table details. To learn more about constructing the dictionary, see table_details.

collection_table_details(table_detail: dict)

item_table_details(table_detail: dict)

class insupdel4stac.Updater(collections_list: list, service_type: str, table_details: list | None = None, loader: Loader | None = None, pgstacdb: PgstacDB | None = None, pgstac_properties: dict | None = None, stacapi_properties: dict | None = None, logger_properties: dict | None = None)

Bases: object

This is a class for updating items and collections in pgSTAC and STAC-API.

Parameters:

• collections_list (list) – A list of collections to be updated.

• service_type (str) – The service type to be used for updating. It can be pgstac or stacapi.

• table_details (list, optional) – A list of dictionaries that contains the details of the tables. Defaults to None.

• loader (Loader, optional) – The loader object to be used for updating.

• ppgstacdb (PgstacDB, optional) – The pgstacdb object to be used for updating.

• pgstac_properties (dict, optional) – The properties of pgstac. Defaults to None.

• stacapi_properties (dict, optional) – The properties of stacapi. Defaults to None.

• logger_properties (dict, optional) – The properties of logger. Defaults to None.

pgstac_collection_update(collection_self_path: str, loader: Loader)

This function updates the collections in pgSTAC.

pgstac_item_update(item: Item, PgstacDB: PgstacDB)

This function updates the items in pgSTAC.

stacapi_collection_update(collection: Collection)

This function updates the collections in STAC-API via PUT method.

stacapi_item_update(item: Item)

This function updates the items in STAC-API via PUT method.

update(collection: Collection, service: str, loader: Loader | None = None, table_details: list | None = None)

This function updates the items and collections in pgSTAC and STAC-API according to the service_type and table_details.

class insupdel4stac.Verifier

Bases: object

A class is implemented to validate the logger_properties prior to use them in the primary source code.

logger_properties(logger_properties: dict) → dict

1.1. Subpackages

• 1.1.1. insupdel4stac.analysers package
  • 1.1.1.1. Submodules
    • 1.1.1.1.1. insupdel4stac.analysers.existance_validator module
      • ExistenceValidator
    • 1.1.1.1.2. insupdel4stac.analysers.properties_verifier module
      • Verifier
    • 1.1.1.1.3. insupdel4stac.analysers.table_details module
      • TableDetails
• 1.1.2. insupdel4stac.statics package
  • 1.1.2.1. Submodules
    • 1.1.2.1.1. insupdel4stac.statics.constants module

1.2. Submodules

• 1.2.1. insupdel4stac.connector module
  • Connector
    • Connector.PgstacDB
    • Connector.loader
    • Connector.pgstac_connection()
    • Connector.pgstac_properties
    • Connector.properties_config()
    • Connector.service
    • Connector.stacapi_connection()
    • Connector.stacapi_properties
• 1.2.2. insupdel4stac.deleter module
  • Deleter
    • Deleter.delete()
    • Deleter.pgstac_collection_delete()
    • Deleter.pgstac_item_delete()
    • Deleter.stacapi_collection_delete()
    • Deleter.stacapi_item_delete()
• 1.2.3. insupdel4stac.inserter module
  • Inserter
    • Inserter.ingester()
    • Inserter.pgstac_collection_ingest()
    • Inserter.pgstac_item_ingest()
    • Inserter.stacapi_collection_ingest()
    • Inserter.stacapi_item_ingest()
• 1.2.4. insupdel4stac.logger module
  • Logger
    • Logger.Null_Handler()
    • Logger.logger_properties
• 1.2.5. insupdel4stac.main module
  • InsUpDel4STAC
    • InsUpDel4STAC.action
    • InsUpDel4STAC.default_catalog_name
    • InsUpDel4STAC.dispatchCollectionActions()
    • InsUpDel4STAC.loader
    • InsUpDel4STAC.logger_properties
    • InsUpDel4STAC.pgstac_properties
    • InsUpDel4STAC.pgstacdb
    • InsUpDel4STAC.service_type
    • InsUpDel4STAC.stac_dir
    • InsUpDel4STAC.stacapi_properties
    • InsUpDel4STAC.table_details
• 1.2.6. insupdel4stac.updater module
  • Updater
    • Updater.pgstac_collection_update()
    • Updater.pgstac_item_update()
    • Updater.stacapi_collection_update()
    • Updater.stacapi_item_update()
    • Updater.update()