PyFunceble.dataset package
Subpackages
Submodules
PyFunceble.dataset.base module
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides the base of all datasets classes.
- Author:
Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- Contributors:
- Project link:
- Project documentation:
- Project homepage:
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
- class PyFunceble.dataset.base.DatasetBase[source]
Bases:
object
Provides the base of all dataset.
- DOWNLOADER: DownloaderBase | None = None
- ensure_source_file_exists()[source]
Ensures that the source file exists before running the decorated method.
- Raises:
ValueError – When
self.source_file
is empty.
- get_content() dict | None [source]
Provides the cached or the real contend of the dataset (after caching)
- Raises:
FileNotFoundError – When the declared file does not exists.
PyFunceble.dataset.csv_base module
PyFunceble.dataset.db_base module
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides the base of all datasets which acts as database interface.
- Author:
Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- Contributors:
- Project link:
- Project documentation:
- Project homepage:
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
- class PyFunceble.dataset.db_base.DBDatasetBase(*, authorized: bool | None = None, remove_unneeded_fields: bool | None = None)[source]
Bases:
DatasetBase
Provides the base of all datasets which acts as database interface.
- add(row: dict) DBDatasetBase [source]
Adds the given dataset into the database.
- Parameters:
row – The row or dataset to add.
- Raises:
TypeError – When the given
row
is not a :py:class`dict`.
- are_equal(read_row: dict, row: dict) bool [source]
Compares the given
read_row
to the row.- Parameters:
read_row – The row read from the dataset infrastructure.
row – The row given by the testing infrastructure.
- cleanup() DBDatasetBase [source]
Cleanups the dataset.
- execute_if_authorized()[source]
Executes the decorated method only if we are authorized to process. Otherwise, apply the given
default
.
- exists(row: dict) bool [source]
Checks if the given dataset exists in our dataset.
- Parameters:
row – The row or dataset to add.
- get_content() Generator[dict | None, None, None] [source]
Provides a generator which provides the next line to read.
- get_filtered_content(filter_map: dict) Generator[dict | None, None, None] [source]
Provides a generator which provides the next dataset. to read.
- get_filtered_row(row: dict) dict [source]
Removes all unkowns fields (not declared) from the given row.
- Parameters:
row – The row to work with.
- remove(row: dict) DBDatasetBase [source]
Removes the given dataset from the database.
- Parameters:
row – The row or dataset to remove.
- Raises:
TypeError – When the given
row
is not a :py:class`dict`.
- property remove_unneeded_fields: bool | None
Provides the current state of the
_remove_unneeded_fields
.
- set_authorized(value: bool) DBDatasetBase [source]
Sets the value of the
_authorized
attribute.- Parameters:
value – The value to set.
- set_remove_unneeded_fields(value: bool) DBDatasetBase [source]
Sets the value of the
_remove_unneeded_fields
attribute.- Parameters:
value – The value to set.
- update(row: dict, *, ignore_if_exist: bool = False) DBDatasetBase [source]
Adds the given dataset into the database if it does not exists. Update otherwise.
- Parameters:
row – The row or dataset to manipulate.
ignore_if_exist – Ignores the insertion/update if the row already exists.
- Raises:
TypeError – When the given
row
is not a :py:class`dict`.
PyFunceble.dataset.iana module
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides an interface which let us interact with the IANA database.
- Author:
Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- Contributors:
- Project link:
- Project documentation:
- Project homepage:
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
- class PyFunceble.dataset.iana.IanaDataset[source]
Bases:
DatasetBase
Provides the dataset handler for the IANA dataset.
- DOWNLOADER: IANADownloader = <PyFunceble.downloader.iana.IANADownloader object>
PyFunceble.dataset.ipv4_reputation module
PyFunceble.dataset.public_suffix module
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides an interface which let us interact with the Public Suffix List.
- Author:
Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- Contributors:
- Project link:
- Project documentation:
- Project homepage:
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
- class PyFunceble.dataset.public_suffix.PublicSuffixDataset[source]
Bases:
DatasetBase
Provides the dataset handler for the Public Suffix List dataset.
- DOWNLOADER: PublicSuffixDownloader = <PyFunceble.downloader.public_suffix.PublicSuffixDownloader object>
PyFunceble.dataset.sql_base module
PyFunceble.dataset.user_agent module
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides an interface which let us interact with the Public Suffix List.
- Author:
Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- Contributors:
- Project link:
- Project documentation:
- Project homepage:
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
- class PyFunceble.dataset.user_agent.UserAgentDataset[source]
Bases:
DatasetBase
Provides the dataset and infrastructure for the User Agent navigation
- DOWNLOADER: UserAgentsDownloader = <PyFunceble.downloader.user_agents.UserAgentsDownloader object>
- get_latest() str [source]
Provides the latest user agent for the given platform.
- Side Effect:
It tries to get the platform and browser from the configuration (if exists).
- is_supported(browser_short_name: str, platform: str) bool [source]
Checks if the given browser and platform is supported.
- is_supported_browser(browser_short_name: str) bool [source]
Checks if the given browser is supported.
- set_prefered(browser_short_name: str, platform: str) UserAgentDataset [source]
Sets the preferred browser to work with.
- Parameters:
browser_short_name – The name of the browser to select.
- Pram platform:
The name of the platform to select.
- Raises:
ValueError – When the given
value
is not supported.
- set_preferred(browser_short_name: str, platform: str) UserAgentDataset [source]
Sets the preferred browser to work with.
- Parameters:
browser_short_name – The name of the browser to select.
- Pram platform:
The name of the platform to select.
- Raises:
ValueError – When the given
value
is not supported.
Module contents
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides all the dataset interaction submodules
- Author:
Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- Contributors:
- Project link:
- Project documentation:
- Project homepage:
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023, 2024 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.