Welcome to nyris' API Documentation. nyris is a high performance visual product search, object detection and visual recommendations engine for retail and industry.
This documentation covers our visual product search engine and the visual recommendations engine.
The implementation in your website or app comprises of 2 steps.
1. Submit your product & image data
In order to recognize your products, we need to know your products. The product feed section covers all details how to submit your product data to us and the specifications for your product or object images.
2. Implement nyris API in you app, website or messenger bot
The nyris engine can be integrated in your app, website, messenger bots, BI solutions - wherever you want to make your customers happy or optimize your data management process. We provide you with code examples how to query our API or ready to go SDKs for iOS and Android.
To get started, you can signup at https://nyris.io/pricing/ or email us at support@nyris.io. Our team will provision Client ID and Client Secret to access our API. Feel free to always contact us should you have questions or suggestions for our service or this documentation.
Thank You & happy coding!
Markus (CTO / Founder)
There are several ways to submit your product and image data to us. We can import nearly every structured data format (xml, csv, txt, json, etc.). If you are already providing your data to an affiliate network like Affilinet, Awin or Google Shopping, you are probably all set for the start. In this case we just need access to your feed URL. Please be aware that those feeds usually only include the link to the main product image. As image recognition works best the more images of a specific product we have, you might need to adjust those feeds to include all your product images. Please find more details below.
If you don't have a product feed, we can also directly access your product data trough an API for the following systems:
Shop Systems.
PIM
You can also create a data feed using Google Sheets. Details on how to use Goole Sheets and an example file can be found in a section below.
The bare minimum is a unique ID (sku) your merchantID (which will be provided to you after signup) and the main_image_link. To get a full list of possible attributes please download the following sheet or use a copy of it as base for your item or product feed. When gathering your data, please keep always in mind that all submitted information will be used for improving your search results. Completeness and quality of data often makes the difference:
Download Feed Example and full attribute description (google sheets)
Attribute | Example | Comment |
---|---|---|
sku |
sku1 | unique item or product identifier (e.g. EAN code) |
merchant_id |
590c4ff62059290e58b35cd1 | your merchantID (will be provisioned to you after signUp) |
main_image_link |
http://cdn.nyris.io/image.jpg | main image link. should be publicly available. ideally, product in question should be at the center of the image |
Image Links
To improve the search, additional images can be provided.
Attribute | Example | Comment |
---|---|---|
additional_image_link_0 |
http://cdn.nyris.io/image_a1.jpg | additional images |
additional_image_link_1 |
http://cdn.nyris.io/image_a2.jpg | additional images |
recognition_image_link_0 |
http://cdn.nyris.io/image_r1.jpg | additional recognition images (images that will never show up on frontend) ideally taken from different angles and w/o packaging |
recognition_image_link_1 |
http://cdn.nyris.io/image_r2.jpg | additional recognition images (images that will never show up on frontend** ideally taken from different angles and w/o packaging |
Geolocation Fields
If provided, the coordinates can be used to narrow down the search by the location of the product. The provided coordinates have to be in the WGS84 format with signed decimal numbers as the notation.
Attribute | Example | Comment |
---|---|---|
latitude |
52.5200 |
Latitude of the product. Use together with
longitude
to associate the product with a location. |
longitude |
13.4050 |
Longitude of the product. Use together with `latitude** to associate the product with a location. |
Other Optional Fields
Attribute | Example | Comment |
---|---|---|
title |
title 1 | Product or item title (supports OCR recognition step) |
brand |
brand 1 | Brand (supports brand recogntion step) |
description_short |
description short 1 | Short description (supports OCR recognition step) |
description_long |
description long 1 | Long description (supports OCR recognition step) |
main_offer_link |
http://www.nyris.io/1 | product or landing page link |
mobile_offer_link |
http://m.nyris.io/landing | secondary product or landing page link (mobile, video etc.) |
availability |
in stock | valid values: in stock;out of stock;preorder;back in stock soon;removed |
price |
17.30 EUR | . as decimal, 3-digit ISO Code for currency |
sale_price |
15.00 EUR | . as decimal, 3-digit ISO Code for currency |
catalog_number_0 |
3306 | catalog numbers from your printed advertisment. Used by PDF importer |
catalog_number_1 |
cat1 | catalog numbers from your printed advertisment. Used by PDF importer |
keyword_0 |
keyword 0 | keyword (supports OCR recognition step) |
keyword_1 |
keyword 1 | keyword (supports OCR recognition step) |
custom_id_key_0 |
pzn | custom ID key (e.g. for Barcode recognition) |
custom_id_value_0 |
123456 | custom ID value (e.g. for Barcode recognition) |
custom_category |
cat1 > cat2 | your product or item category. Please separate with > (e.g. building materials, wood, windows & doors > windows > window ledges). Providing this field will be used to improve recommendations. |
group_id |
23 | for variation products (size, colour etc. variations) |
feed_id |
1 | Feed ID, required only if you have multiple feeds as one merchant. The value has to be an integer. |
language |
de-DE | Language Culture Name ( https://msdn.microsoft.com/de-de/library/ee825488(v=cs.20).aspx ) |
meta_data |
some additional data | any data you would like to store additionally |
Send an image to the server. The server will process the image and return a JSON object with default values or with the values defined in your template. Use the provided API-Key for authentication.
Headers
Key | Value |
---|---|
X-Api-Key |
The API key provided by us. Preferred method |
Authorization |
Bearer + 'access_token'.
Deprecated to X-Api-Key |
Content-Type |
image/jpeg |
Content-Length |
the size of image in bytes |
Body
The body of the request should contain the image file's actual bytes sent in binary.
General Note on Images
OkHttpClient client = new OkHttpClient();
Request request = new Request.Builder()
.url("https://api.nyris.io/find/v1")
.post(null)
.addHeader("x-api-key", "KEY")
// or instead of the api-key .addHeader("authorization", "Bearer access-token")
.addHeader("content-type", "image/jpeg")
.addHeader("content-length", "image-file-size-in-bytes")
.addHeader("cache-control", "no-cache")
.build();
Response response = client.newCall(request).execute();
{
"results": [
{
"sku": "123454643",
"score": 0.96
},
{
"sku": "123454644",
"score": 0.92
},
{
"sku": "123454645",
"score": 0.88
},
{
"sku": "123454646",
"score": 0.83
}
]
}
There are additional header attributes you can use to change the results. Please find the overview below
In case you are providing us two different feeds for the same products using different languages, you can use the Accept-Language header to filter the results by a specific language.
Header | Example Value |
---|---|
Accept-Language |
de-CH |
Accept-Language |
fr-CH |
If you don't set the Accept-Language header, the results for all languages will be returned.
If you want to get all available fields, you can use the Accept
Header as following:
Header | Example Value |
---|---|
Accept |
application/offers.complete+json |
Note, that you need to provide us the appropriate data in your product data feed, so we can display them. Fields which have no value will not be included in the JSON response.
{
"results": [{
"title": "test product",
"descriptionShort": "Lorem ipsum dolor sit amet, consetetur sadipscing elitr.",
"descriptionLong": "Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat.",
"language": "de-DE",
"brand": "test brand",
"catalogNumbers": ["test_catalog_number_0",
"test_catalog_number_1"],
"customIds": {
"DEMO_CUSTOM_ID": "DEMO_ID_123"
},
"keywords": ["demo",
"demonstration",
"test"],
"categories": ["Demo",
"Test"],
"availability": "in stock",
"feedId": "1",
"groupId": "1",
"price": "10.00 EUR",
"salePrice": "1.00 EUR",
"links": {
"main": "https://www.mingaberlin.com/de/bunte-socken-muster-sliced-sunburst.html",
"mobile": "https://www.mingaberlin.com/media/catalog/product/cache/2/thumbnail/100x100/0dc2d03fe217f8c83829496872af24a0/0/0/0086.jpg"
},
"images": ["https://img.staging.everybag.net/809/809450e02adc0a9bd52a19f6a212307ad9803323b5c42ab9b4d29eb27fa73b7f.jpg",
"https://img.staging.everybag.net/2b4/2b4f7ac06c51e55b8de382667a45bfa37082c87f019be7848d8abbbc4869acb5.jpg"],
"metadata": "This product is for demonstrations only",
"sku": "123456",
"score": 1
}]
}
You can control general search modes and parameters such as matching type,
result limits, thresholds etc. using the X-Options
header.
Modes can be used exclusively (by stating their name
as-is) or added to
(+name
) and removed from (-name
) the options, wheras parameters are
specified as key=value
pairs. The values should be separated by a space character, e.g. X-Options: limit=10 regoup
.
Core Search Modes | Explanation |
---|---|
default |
The default search mode, implies
exact +similarity +ocr limit=20
(see below). |
exact |
Performs exact matching, e.g. for search on product packages and catalog pages or generally whenever you know what specific instance in the data feed your are searching for. |
similarity |
Performs similarity matching, e.g. for search on items that look like, but not exactly like, the content of the request image. |
ocr |
Enables optical character recognition on the images and uses this to search in text fields of the data feed, such as title and description. |
Optional Search Modes | Explanation |
---|---|
regroup |
If your data feed contains varations on the same item but lacks a group ID value, this mode enables regrouping of the items. Since this overwrites the original group ID information and incurs additional processing overhead, it should only be used when needed. |
objects |
Enables object detection before using similarity matches. |
barcode |
Enables the barcode detection. |
recommendations |
Forces recommendation-type searches that don't early-exit in presence of a perfect result and always return indicative results. |
ocr.supress-numbers |
Determines if numbers should be filtered out from the ocr search result. |
Parameter | Example Value | Explanation |
---|---|---|
limit |
limit=20
(default)
, integers between 1 and 100 |
The upper limit for the number of results to be returned. |
threshold |
threshold=0.8 |
The final threshold to apply to the scores. Only results above or equal to that score will be returned. |
category-prediction.limit |
category-prediction.limit=3 |
Limits the number of categories to return (e.g. the top
3
; defaults to all categories). |
category-prediction.mode |
category-prediction.mode=1 |
Selects the score combination mode, default is Lukasson, 0 = Softmax, 1 = Lukasson, 2 = AveragePrecision. |
category-prediction.threshold |
category-prediction.threshold=0.5 |
Sets the cutoff threshold for category predictions (range
0..1
). |
category-prediction.weighted |
category-prediction.weighted=1 |
Controls weighting (
0
disabled,
1
enabled (default)). |
exact.limit |
exact.limit=10 |
Limits the number of exact results. |
exact.threshold |
exact.threshold |
Sets the lower cutoff value for results from MESS. The value can be between
0.0
and
1.0
. |
exact.threshold.indicative |
exact.threshold.indicative=0.85 |
Sets the lowest cutoff; every score below this is considered a bad match. |
exact.threshold.perfect |
exact.threshold.perfect=0.95 |
Sets the highest cutoff; every score above this is considered a perfect match. |
ocr.threshold |
ocr.threshold=0.8 |
Sets the lower cutoff value for results from ocr. The lowest possible value is
0
. |
ocr.threshold.indicative |
ocr.threshold.indicative=0.5 |
Sets the lowest cutoff; every score below this is considered a bad match. |
ocr.threshold.perfect |
ocr.threshold.perfect=0.8 |
Sets the highest cutoff; every score above this is considered a perfect match. |
similarity.limit |
similarity.limit=20
(default)
, real between 1 and 100 |
The upper limit for the number of results to be returned from
similarity
mode searches. Note that this maily affects
similarity
-only search, as combined searches will be ultimatively restricted by the
limit
parameter. |
similarity.skip |
similarity.skip=1 |
Skips the first
N
results returned from MESS, if they exceed the configured threshold. |
similarity.skip.threshold |
similarity.skip.threshold=0.99 |
Sets the lower cutoff value for result skipping from similarity. The value can be between
0.0
and
1.0
and defaults to a value close to
1.0
. |
similarity.threshold |
similarity.threshold=0.7
, real between 0 and 1 |
The lower limit of confidences to be considered good from
similarity
mode matches. |
similarity.threshold.indicative |
similarity.threshold.indicative=0.7 |
Sets the lowest cutoff; every score below this is considered a bad match. |
similarity.threshold.perfect |
similarity.threshold.perfect=0.9 |
Sets the highest cutoff; every score above this is considered a perfect match. |
regroup.threshold |
regroup.threshold=0.9
, real between 0 and 1 |
The lower limit of confidences for the
regroup
mode. |
Here are some default options of interest:
Core Search Modes | Explanation |
---|---|
X-Options: limit=10 |
Returns at most
10
results. |
X-Options: exact limit=1 |
Enables exact matches only and returns exactly
0
or
1
result. Use this for high-performance cover or poster searches or similar use cases. |
X-Options: -ocr |
Performs an otherwise default search but disables text recognition. Same as
X-Options: default -ocr
. |
X-Options: similarity similarity.limit=10 +regroup |
Performs similarity search only with at most
10
results and requests group ID generation for the results. |
If products in the feed are provided with geolocation, the information can be used to narrow down specifying a maximal distance between the user and the product. The provided coordinates have to be in the WGS84 format with signed decimal numbers as the notation. The following query parameters are required:
Name | Example | Explanation |
---|---|---|
lat |
52.5200 |
The latitude of the user. |
lon |
13.4050 |
The longitude of the user. |
dist |
10000 |
The distance from the center in meter. |
MESS is our ultra fast visual similarity search algorithm. It is based on the semantic content of an image and can be used to help you categorize your products or to create a real time visual recommendation engine on your website. It uses the same image data that you have submitted to us for the regular visual search, so there is nothing for you to change.
To see MESS in action, please check out the nyris MESS Site
To use the MESS search only, all you have to do is to add the X-Options: similarity
header to your request. There is no value required.
Header | Example Value |
---|---|
X-Options |
similarity |
What is the difference between MESS and the nyris visual product search?
MESS is just a single search algorithm while our general visual product search uses different technologies to find the best product match. Because MESS search is based only on the semantic content of the image, the response times are very fast and the search can be executed on page load. MESS was designed to work especially well with professional studio product images, which are usually used on e-commerce websites.
OkHttpClient client = new OkHttpClient();
Request request = new Request.Builder()
.url("https://api.nyris.io/find/v1")
.post(null)
.addHeader("x-api-key", "KEY")
// or instead of the api-key .addHeader("authorization", "Bearer access-token")
.addHeader("content-type", "image/jpeg")
.addHeader("content-length", "image-file-size-in-bytes")
.addHeader("cache-control", "no-cache")
.addHeader("X-Options", "similarity")
.build();
Response response = client.newCall(request).execute();
{
"results": [
{
"sku": "123454643",
"score": 0.96
},
{
"sku": "123454644",
"score": 0.92
},
{
"sku": "123454645",
"score": 0.88
},
{
"sku": "123454646",
"score": 0.83
}
]
}
To get visually similar items for a specific product in your catalogue, you can also submit just the SKU of this product to us. As we already know this product (because we imported it for you) we perform the visual similarity search based on the main image of this product.
To use the visual recommendation engine with your SKU please send a simple GET request with your SKU to the following endpoint:
API Endpoint (GET)
https://api.nyris.io/recommend/v1/
Example for SKU 03764560:
https://api.nyris.io/recommend/v1/03764560
The response will not include the product with the SKU you have send with the request. Although this would be the most similar product, it's the same product as the product you have send the request for, so it's removed from the results set.
{
"results": [
{
"sku": "123454643",
"score": 0.96
},
{
"sku": "123454644",
"score": 0.92
},
{
"sku": "123454645",
"score": 0.88
},
{
"sku": "123454646",
"score": 0.83
}
]
}
This option is deprecated in favor of using the API-Key.
Once you have the Client ID and Client Secret, you have to generate an access token to use our API end points. Generating a token requires a POST request to the OPEN ID Server with an URL encoded Form.
Property | Description |
---|---|
grant_type |
password or client_credentials |
username |
required if grant_type is password |
password |
required if grant_type is password |
client_id |
required |
client_secret |
required |
scope |
image_matching |
OkHttpClient client = new OkHttpClient();
MediaType mediaType = MediaType.parse("application/x-www-form-urlencoded");
RequestBody body = RequestBody.create(mediaType, "grant_type=client_credentials&client_id=your_client_id&client_secret=your_client_secret&scope=image_matching");
Request request = new Request.Builder()
.url("https://api.nyris.io/connect/token")
.post(body)
.addHeader("content-type", "application/x-www-form-urlencoded")
.addHeader("cache-control", "no-cache")
.build();
Response response = client.newCall(request).execute();
{
"token_type": "Bearer",
"access_token": "your-unique-access-token",
"expires_in": 2592000
}
The fastest and easiest way to integrate visual search to your website is using our Javascript WebApp. All you have to do is to add 3 lines of code to the header or footer section of your website and you are ready to start testing.
What I need to start using this code?
All you need is your API key.
How to use the code?
To use the code you need to reference a CSS file, a JS file, and declare a global variable that holds your API key.
Variation 1 - add all files in one place
Inside your HTML (best just before the closing
tag) add the following three lines of code:
<link href="https://assets.nyris.io/libs/js/1.1/search/nyris.min.css" rel="stylesheet" type="text/css" />
<script>var gNyrisApiKey='<YOUR API KEY HERE>';</script>
<script src="https://assets.nyris.io/libs/js/1.1/search/nyris.min.js" type="text/javascript"></script>
Variation 2 - CSS in Header / JS in Footer
Inside your HTML head element (just before ) add this line to load the CSS:
<link href="https://assets.nyris.io/libs/js/1.1/search/nyris.min.css" rel="stylesheet" type="text/css" />
Just before the end of your HTML body element (just before ) add these lines:
<script>var gNyrisApiKey='<YOUR API KEY HERE>';</script>
<script src="https://assets.nyris.io/libs/js/1.1/search/nyris.min.js" type="text/javascript"></script>
Add your API key
Replace
Done. You are ready to use visual search on your website!
Sample
There is a sample page here where you can enter your API key and see the widget in action
https://assets.nyris.io/libs/js/1.1/search/sample.html
Where is the source code? Can I change it
The source files can be downloaded here:
https://assets.nyris.io/libs/js/1.1/search/source.tar.gz
You can change or adjust the JS / CSS files according to your needs and host them on your server as well.
You can find the latest Android SDK including the implementation documentation on github:
You can find the latest iOS SDK including the implementation documentation on github:
https://github.com/nyris/Nyris.IMX.iOS
Should you have any questions, please send and e-mail to support@nyris.io
Visual search analytics provide valuable insight into your customers search behavior and at the same time give us valuable input to improve the visual search performance. There are two kind of Analytics that we offer
The general search analytics will provide you
In order to improve our visual search, we need to know what happens after we delivered the visual search results. Each request we deliver to you has a unique request_id
found in the response header. Using the request_id
and our feedback API, you can submit multiple events to our analytics engine. Below you will find the list of events and the API Endpoint including an example how to use it.
In order to implement click and conversion analytics on your site, you need to send nyris all click and conversion events to the feedback endpoint (POST to https://api.nyris.io/feedback/v1):
Please use the provided API-Key in the header for authentication.
Authentication Header
Key | Value |
---|---|
X-Api-Key |
The API Key provided by us. |
The server accepts only valid JSON documents, content-Type: text/json, application/json or application/event+json
The submitted JSON document has to include the
request_id
timestamp
event
(event type)data
(event type data, optional for some event types)session_id
(optional, the first requestID for the same user. The session_id
can group multiple requests from a single user.)The possible event types are:
event | comment |
---|---|
click |
which position in the list and/or which specific product id was clicked |
conversion |
a merchant defined conversion event |
feedback |
customer feedback on search results |
region |
selected region in submitted image (when using object proposal) |
Tracks the position and the ID of the clicked item in the results set:
event
: clickdata
: positions (array, int16), product_ids (array, string)positions
or product_ids
are optional (one of both is required). The product_id has to be the same product_id as in your product data so we can link those in the analytics engine.
{
"request_id": "e8848827-27af-41d7-a98f-1a2f94b0f19a",
"timestamp": "2012-04-23T18:25:43.511Z",
"session_id": "e8848827-27af-41d7-a98f-1a2f94b0f19a",
"event": "click",
"data": {
"positions": [10],
"product_ids": ["1234"]
}
}
Tracks any merchant defined conversion (e.g. add to cart, sale, view)
event
: conversiondata
: positions (array, int16, optional), product_ids (array, string, optional)positions
and product_ids
are optional. Sending the conversion event without any data will work as well.
{
"request_id": "e8848827-27af-41d7-a98f-1a2f94b0f19a",
"timestamp": "2012-04-23T18:25:43.511Z",
"session_id": "e8848827-27af-41d7-a98f-1a2f94b0f19a",
"event": "conversion",
"data": {
"positions": [10],
"product_ids": ["1234"]
}
}
Tracks customer feedback on search results
event
: feedbackdata
: success (boolean), comment (string, optional){
"request_id": "e8848827-27af-41d7-a98f-1a2f94b0f19a",
"timestamp": "2012-04-23T18:25:43.511Z",
"session_id": "e8848827-27af-41d7-a98f-1a2f94b0f19a",
"event": "feedback",
"data": {
"success": true,
"comment": "Thank you!"
}
}
Tracks the customer selected image region
event
: regiondata
: form & coordinatesrect
: The normalized region rectangle that is sentx
: The left coordinate of the region as a fraction of the image width (range 0..1)y
: The top coordinate of the region as a fraction of the image height (range 0..1)w
: The width of the region as a fraction of the image width (range 0..1)h
: The height of the region as a fraction of the image height (range 0..1){
"request_id": "e8848827-27af-41d7-a98f-1a2f94b0f19a",
"timestamp": "2012-04-23T18:25:43.511Z",
"session_id": "e8848827-27af-41d7-a98f-1a2f94b0f19a",
"event": "region",
"data": {
"rect": {
"x": 0.25,
"y": 0.25,
"w": 0.5,
"h": 0.5
}
}
}