Problems connecting to remote YUM package repositories

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Problems connecting to remote YUM package repositories

icycle
Hi,
I'm trying to setup "Artifactory Pro Power Pack 3.9.0 (rev. 30199)" for Remote Repository connections to some YUM repo URLs that work fine with YUM directly but get "Connection failed: Error 404: Not Found" errors when testing with AF. None of the following baseurl's work, and these repo definitions are straight out of a working yum file:

[elasticsearch-2.x]
name=Elasticsearch repository for 2.x packages
baseurl=https://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

[logstash-2.2]
name=Logstash repository for 2.2.x packages
baseurl=http://packages.elastic.co/logstash/2.2/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

[kibana-4.4]
name=Kibana repository for 4.4.x packages
baseurl=http://packages.elastic.co/kibana/4.4/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

[beats]
name=Elastic Beats Repository
baseurl=https://packages.elastic.co/beats/yum/el/$basearch
enabled=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
gpgcheck=1

It appears that elastic.co may host these repos on S3, not sure if that's correct or if it matters. But they are also not web-browsable either.

Am I doing something wrong? Do I need to upgrade AF?

Thanks,
Brett
Reply | Threaded
Open this post in threaded view
|

Re: Problems connecting to remote YUM package repositories

icycle
Still blocked on this. Without resolution it's not looking like AF remote repos will be of much use to our organization. We've also tried a test instance of AF 4.x and it shows the same issue.

I've tried testing various levels of the https://packages.elastic.co/ directory hierarchy with AF's remote repo "test" button. The only one that AF accepts as legit is the top level URL itself. However, this is rejected by yum itself:

# repoquery --repofrompath=elktop,https://packages.elastic.co/ --repoid=elktop -q -a
failure: repodata/repomd.xml from elktop: [Errno 256] No more mirrors to try.
https://packages.elastic.co/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found

and, when that AF remote repo is queried by yum, it too fails:
# repoquery --repofrompath=testelk,http://usr:pw@af-server:8081/artifactory/simple/elk-remote --repoid=testelk -q -a
failure: repodata/repomd.xml from testelk: [Errno 256] No more mirrors to try.
http://usr:pw@af-server:8081/artifactory/simple/elk-remote/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

For comparison, here's a very specific repo baseurl that works for yum but is rejected by AF's test button with a 404 error:
[root@ip-10-4-131-34 yum.repos.d]# yum --disablerepo='*' --enablerepo=beats repoinfo
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Repo-id      : beats/x86_64
Repo-name    : Elastic Beats Repository
Repo-revision: 1463586608
Repo-updated : Wed May 18 15:50:10 2016
Repo-pkgs    : 38
Repo-size    : 138 M
Repo-baseurl : https://packages.elastic.co/beats/yum/el/x86_64/
Repo-expire  : 21,600 second(s) (last: Mon Jun  6 17:07:04 2016)
Repo-filename: /etc/yum.repos.d/elk.repo

repolist: 38

[root@ip-10-4-131-34 yum.repos.d]# yum --disablerepo='*' --enablerepo=beats list available
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Available Packages
filebeat.x86_64                                                                  1.2.3-1                                                                 beats
packetbeat.x86_64                                                                1.2.3-1                                                                 beats
topbeat.x86_64                                                                   1.2.3-1                                                                 beats

I went ahead and added the remote repo with the same URL as above anyway and tried it:
# repoquery --repofrompath=elkbeats,http://usr:pw@af-server:8081/artifactory/simple/elk-beats --repoid=elkbeats -q -a
failure: repodata/repomd.xml from elkbeats: [Errno 256] No more mirrors to try.
http://usr:pw@af-server:8081/artifactory/simple/elk-beats/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden

Given this new info that shows a specific 403 error rather than the 404 error exposed by the test button, a fresh google shows what may be the issue:
https://www.jfrog.com/jira/browse/RTFACT-7966?focusedCommentId=37388&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-37388

Unsure what next steps are. I will try contacting the elastic.co folks.

Thanks for any tips here,
Brett
Reply | Threaded
Open this post in threaded view
|

Re: Problems connecting to remote YUM package repositories

icycle
In reply to this post by icycle
I believe we've figured this out. It's a combination of two issues:

1) The S3 bucket(s) hosting the elastic.co repos apparently do not have the s3:ListBucket permission, which causes the AF "test" button to fail (with a 404 error) as well as attempts to browse to elastic.co repo paths in the browser.

2) Even though the AF "test" fails, the remote repo definition can still be saved. At that point, however, testing it showed a 403 error which I thought also came from S3. But, the 403 in fact came from AF as a result of the AF user not having deploy permissions on that remote repo. Once this permission was added, the remote repo pointing to one of elastic.co's repos began working!

Just recording the final outcome here for posterity...