An API to search public repositories

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

An API to search public repositories

Evgeny Goldin

Hi,

I was just wondering how a lookup between all public repositories happens?
Is there an API each repository responds to? Or one (in our case,
Artifactory) just attempts to download and see if it fails or not ?

--
View this message in context: http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24302503.html
Sent from the Artifactory-Users mailing list archive at Nabble.com.


------------------------------------------------------------------------------
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users
Best regards,
Evgeny
http://evgeny-goldin.com/
Reply | Threaded
Open this post in threaded view
|

Re: An API to search public repositories

Evgeny Goldin


Does it have anything to do with  http://maven.apache.org/wagon/ Maven Wagon
?


Evgeny Goldin wrote:
>
> Hi,
>
> I was just wondering how a lookup between all public repositories happens?
> Is there an API each repository responds to? Or one (in our case,
> Artifactory) just attempts to download and see if it fails or not ?
>
>

--
View this message in context: http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24310068.html
Sent from the Artifactory-Users mailing list archive at Nabble.com.


------------------------------------------------------------------------------
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users
Best regards,
Evgeny
http://evgeny-goldin.com/
Reply | Threaded
Open this post in threaded view
|

Re: An API to search public repositories

Yoav Landman
Administrator
In reply to this post by Evgeny Goldin
Hi Evgeny,

When Artifactory contacts a remote repository it first sends a HEAD request to see whether the artifact exists and if it has changed (changes are only checkedĀ  for snapshots - the maven metadata is queried). Even though there are more powerful search APIs provided by repo managers, including Artifactory itself, this is the fastest direct query form, which is standard and works with 'dumb' HTTP repositories as well (in fact, HTTP also offers conditional-GET in a single request, but repo1 does not support it, so we use a HEAD then GET). Other than that, each remore repository has an expiring cache for snapshots, a cache for failed retrievals (network errors etc.) and a cache for non-found artifacts (all configurable from the remote repo advanced configuration panel).
The actual resolution order between repositories is explained here: http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories (the last section called "Resolution Order").

HTH,

Yoav

On Thu, Jul 2, 2009 at 10:34 AM, Evgeny Goldin <[hidden email]> wrote:

Hi,

I was just wondering how a lookup between all public repositories happens?
Is there an API each repository responds to? Or one (in our case,
Artifactory) just attempts to download and see if it fails or not ?

--
View this message in context: http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24302503.html
Sent from the Artifactory-Users mailing list archive at Nabble.com.


------------------------------------------------------------------------------
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users


------------------------------------------------------------------------------

_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users
Reply | Threaded
Open this post in threaded view
|

Re: An API to search public repositories

Evgeny Goldin

Great, Yoav, thanks a lot!


Yoav  Landman wrote:

>
> Hi Evgeny,
>
> When Artifactory contacts a remote repository it first sends a HEAD
> request
> to see whether the artifact exists and if it has changed (changes are only
> checked  for snapshots - the maven metadata is queried). Even though there
> are more powerful search APIs provided by repo managers, including
> Artifactory itself, this is the fastest direct query form, which is
> standard
> and works with 'dumb' HTTP repositories as well (in fact, HTTP also offers
> conditional-GET in a single request, but repo1 does not support it, so we
> use a HEAD then GET). Other than that, each remore repository has an
> expiring cache for snapshots, a cache for failed retrievals (network
> errors
> etc.) and a cache for non-found artifacts (all configurable from the
> remote
> repo advanced configuration panel).
> The actual resolution order between repositories is explained here:
> http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories
> (the
> last section called "Resolution Order").
>
> HTH,
>
> Yoav
>
> On Thu, Jul 2, 2009 at 10:34 AM, Evgeny Goldin <[hidden email]> wrote:
>
>>
>> Hi,
>>
>> I was just wondering how a lookup between all public repositories
>> happens?
>> Is there an API each repository responds to? Or one (in our case,
>> Artifactory) just attempts to download and see if it fails or not ?
>>
>> --
>> View this message in context:
>> http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24302503.html
>> Sent from the Artifactory-Users mailing list archive at Nabble.com.
>>
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Artifactory-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/artifactory-users
>>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Artifactory-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/artifactory-users
>
>

--
View this message in context: http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24340827.html
Sent from the Artifactory-Users mailing list archive at Nabble.com.


------------------------------------------------------------------------------
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users
Best regards,
Evgeny
http://evgeny-goldin.com/
Reply | Threaded
Open this post in threaded view
|

Re: An API to search public repositories

Evgeny Goldin
In reply to this post by Yoav Landman

http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories:

The search order is agnostic to the order in which sub-repositories are
listed inside a virtual repository.

But it's not agnostic to the order remote repositories are
http://img30.imageshack.us/img30/8305/32047237.png listed , right ?

--
View this message in context: http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24340916.html
Sent from the Artifactory-Users mailing list archive at Nabble.com.


------------------------------------------------------------------------------
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users
Best regards,
Evgeny
http://evgeny-goldin.com/
Reply | Threaded
Open this post in threaded view
|

Re: An API to search public repositories

Yoav Landman
Administrator
Right. Updated the doc to reflect that.

Thanks!

On Sun, Jul 5, 2009 at 10:58 AM, Evgeny Goldin <[hidden email]> wrote:

http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories:

The search order is agnostic to the order in which sub-repositories are
listed inside a virtual repository.

But it's not agnostic to the order remote repositories are
http://img30.imageshack.us/img30/8305/32047237.png listed , right ?

--
View this message in context: http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24340916.html
Sent from the Artifactory-Users mailing list archive at Nabble.com.


------------------------------------------------------------------------------
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users


------------------------------------------------------------------------------

_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users
Reply | Threaded
Open this post in threaded view
|

Re: An API to search public repositories

Yoav Landman
Administrator
P.S. This is not true just for remotes, but also applies to the internal order of locals an other virtual repos within a virtual repo - the wiki doc explains this in more detail.

On Sun, Jul 5, 2009 at 9:53 PM, Yoav Landman <[hidden email]> wrote:
Right. Updated the doc to reflect that.

Thanks!


On Sun, Jul 5, 2009 at 10:58 AM, Evgeny Goldin <[hidden email]> wrote:

http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories:

The search order is agnostic to the order in which sub-repositories are
listed inside a virtual repository.

But it's not agnostic to the order remote repositories are
http://img30.imageshack.us/img30/8305/32047237.png listed , right ?

--
View this message in context: http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24340916.html
Sent from the Artifactory-Users mailing list archive at Nabble.com.


------------------------------------------------------------------------------
_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users



------------------------------------------------------------------------------

_______________________________________________
Artifactory-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/artifactory-users