Opendata.eol.org questions and requests

EOL Services
21

Wikidata did not continue with importing vernacular names because the use of capitals in common names are different per taxon/country. I am afraid it is not enough in the end but I will check next week.

22

If wikidata gets an update, will the names in EOL get an update too?

23

Yes, although EOL tends to lag behind Wikidata. Right now we are only updating it every few months; we should be able to pick up the pace once our newer, smarter harvesting code is in place.

:slight_smile:
Jen

24

Hi,

I’m managing to retrieve some traits albeit in a very slipshod fashion.

Here’s what I’m trying to achieve:

  1. Query for a particular taxon to see if it exists in eol.org database e.g.Perodicticus edwardsi
  2. If it does exist, then return all attributes (traits) for that taxon (preferably in key-value json format).

I can do the search with the classic APIs:

eol.org/api/search/1.0.json?q=perodicticus%20edwardsi&page=1&key=<>

But in the results there is no indication of whether a result is a ‘Page’ or ‘Unresolved name’. I of course would need to ensure that I am only dealing with taxa for which there is a Page as this seems the surest way of ensuring I get the train for any particular taxon.

I would like to search for any particular taxon with a Cypher query (if possible), but I know little about Cypher and despite looking through the docs, understand less. Querying databases is a new thing for me.

I thought something like this might work to return the page but it doesn’t:

MATCH (p:Page {canonical: "Perodicticus edwardsi"})
RETURN p.page_id
LIMIT 4

Then, once I have the page_id of that taxon, I would like to do a Cypher query to return all attributes of that taxon as shown on the ‘overview’ tab of any particular Page.

And I would like the returned results to be in key-value-paired JSON just like traditional JSON returned results, not in the 'column-data JSON currently returned by the Cypher queries to EOL.

Or, if returning the JSON in key-value pairs, is not possible, do you know of some repo or algorithm I can use to easily convert it?

I am writing in Swift and Xcode FYI.

Thanks,

Ryan

25

Hi, Ryan! Your first query looks alright to me. When I tried it, it returned

{“columns”:[“p.page_id”],“data”:[[4454403]]}

and that ID looks like a nice rich species page. You could then run something like this:

MATCH (t:Trait)<-[:trait|inferred_trait]-(p:Page {page_id:4454403}),
(t)-[:predicate]->(pred:Term)
OPTIONAL MATCH (t)-[:units_term]->(units:Term)
OPTIONAL MATCH (t)-[:lifestage_term]->(stage:Term)
OPTIONAL MATCH (t)-[:object_term]->(obj:Term)
RETURN pred.name, t.measurement, units.name, stage.name, obj.name
LIMIT 1000

I’m afraid the only formats available are the two described in the documentation. You may find the csv easier to work with if neo4j’s native JSON doesn’t suit you. You’ll get more than two fields, regardless. It’s the data themselves that make it this complicated. Numerical values and categorical values, for instance, are stored differently, because they require different contextual metadata and vocabularies. Also, each record may carry caveats in its metadata that you’ll want to see. Lifestage is one of the most important (eg, for body size attributes) so I included it in the example above. You can also get provenance information for each record, which may be important for your data users.

One last thing: you should anticipate up to hundreds of records for a rich page like a mammal species. Only a sample are shown on the Overview tab. To get an idea what is available, open the data tab.

Good hunting,

Jen

26

Hi Jen,

Thanks for getting back to me. Finally I’m getting around to implementing the EOL data.

As I mentioned previously, I’m currently doing two calls in succession:

  1. The first call gets the page id from a given taxon.
  2. The second call gets the returned traits for the taxon.

I used your suggestion and just wanted to ensure that this does in fact return ALL the traist for the taxon in question:

https://eol.org/service/cypher?utf8=✓
     &query=
     MATCH (t:Trait)<-[:trait|inferred_trait]-(p:Page {page_id:\(page.id)}),
     (t)-[:predicate]->(pred:Term)
     OPTIONAL MATCH (t)-[:units_term]->(units:Term)
     OPTIONAL MATCH (t)-[:lifestage_term]->(stage:Term)
     OPTIONAL MATCH (t)-[:object_term]->(obj:Term)
     RETURN pred.name, t.measurement, units.name, stage.name, obj.name
     LIMIT 10
     &format=cypher

Thanks again,

Ryan
Hong Kong

27

You won’t want to use LIMIT 10 in that case. The number of available records varies, but there may be dozens for voluminous categories like geographic distribution or habitat. LIMIT 1000 should be safe. The only category that query won’t fetch are the ecological association records, which have a different structure. To include those, try adding:

OPTIONAL MATCH (t)-[:object_page]->(p2:Page)

And in the RETURN line (btw, I’d use RETURN DISTINCT) add p2.canonical, or p2.page_id, however you’d like to identify the prey, host, etc. in the ecological association.

Then you should get everything.

:slight_smile:
Jen

28

Hi there,

Sorry if this is something silly on my end, but I’ve been trying to download the active version EOL dynamic hierarchy (https://opendata.eol.org/dataset/0a023d9a-f8c3-4c80-a8d1-1702475cda18/resource/00adb47b-57ed-4f6b-8f66-83bfdb5120e8/download/dh21.zip) and receiving a 503 server error. Is this expected?

Thanks for your help!
Jonah

29

Not silly at all, and I’m sorry for the inconvenience. The product we have been using to manage our downloadables is becoming difficult to maintain. We’re in the process of setting up a replacement, and the files are available now, just not the new interface. The DH version you want is now accessible at https://editors.eol.org/uploaded_resources/00a/db4/7b-57ed-4f6b-8f66-83bfdb5120e8

Thank you for reporting!

Jen

30

Amazing, thanks so much!