Skip to content

Add simple page parsing for full syncs#466

Merged
gerrod3 merged 1 commit into
pulp:mainfrom
gerrod3:pulp-pulp-sync
May 26, 2022
Merged

Add simple page parsing for full syncs#466
gerrod3 merged 1 commit into
pulp:mainfrom
gerrod3:pulp-pulp-sync

Conversation

@gerrod3

@gerrod3 gerrod3 commented Nov 10, 2021

Copy link
Copy Markdown
Contributor

WIP syncing using simple page parsing. Implemented as a fallback for repositories that don't implement the PyPI XMLRPC endpoints (e.g. Pulp). Enables full Pulp to Pulp syncing.

@gerrod3 gerrod3 added this to the 3.6.0 milestone Dec 14, 2021
@gerrod3 gerrod3 linked an issue Dec 14, 2021 that may be closed by this pull request
@gerrod3
gerrod3 marked this pull request as ready for review December 14, 2021 16:19
@gerrod3 gerrod3 removed this from the 3.6.0 milestone Dec 15, 2021
Comment thread docs/tech-preview.rst Outdated
Bandersnatch config file.
* PyPI’s json API at content endpoint ‘/pypi/{package-name}/json’. Allows for basic Pulp-to-Pulp syncing.
* Fully mirror Python repositories like PyPI.
* Fully mirror Python repositories like PyPI and Pulp.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Fully mirror Python repositories like provided by PyPI and Pulp itself.

Comment thread pulp_python/app/tasks/sync.py
Comment thread pulp_python/tests/functional/api/test_download_content.py Outdated
@stale

stale Bot commented May 26, 2022

Copy link
Copy Markdown

This pull request has been marked 'stale' due to lack of recent activity. If there is no further activity, the PR will be closed in another 30 days. Thank you for your contribution!

@stale stale Bot added the stale label May 26, 2022
@stale

stale Bot commented May 26, 2022

Copy link
Copy Markdown

This pull request is no longer marked for closure.

@stale stale Bot removed the stale label May 26, 2022
Comment thread pulp_python/app/tasks/sync.py Outdated
repo3 = self._create_repo_and_sync_with_remote(remote)
self.assertEqual(get_content_summary(repo3.to_dict()), PYTHON_LG_FIXTURE_SUMMARY)

def test_full_fixtures_to_pulp_sync(self):

@dralley dralley May 26, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little curious what happens if you sync an on-demand repo from pulp into pulp, in immediate mode. Would be an interesting test to make sure nothing blows up.

Maybe as part of the pulp_file suite?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although we might already have that test, I haven't looked.

@dralley dralley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor question about the error message which could potentially use improvement, but otherwise this all looks good.

@dralley

dralley commented May 26, 2022

Copy link
Copy Markdown
Contributor

The pulp-to-pulp sync test is failing however ^

@gerrod3

gerrod3 commented May 26, 2022

Copy link
Copy Markdown
Contributor Author

The pulp-to-pulp sync test is failing however ^

Yeah I need to figure out how to properly set the SSL certs in the CI or else the Pulp host won't be trusted.

Edit: NVM, it just required a template settings change.

@gerrod3
gerrod3 merged commit 074142a into pulp:main May 26, 2022
@gerrod3
gerrod3 deleted the pulp-pulp-sync branch May 26, 2022 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pulp to pulp syncs fail with 405 on /pypi

2 participants