Inferring visual semantic similarity with deep learning and Wikidata: Introducing imagesim-353 |
|
| Abstract | Aiming at multi-modal knowledge representation we construct a dataset with pairs of digital photos of objects. We manually score image pairs for semantic object similarity. A pre-trained ImageNet-based deep neural network predicts the objects and we use the output to estimate the similarity between two images. With a linkage between the neural network and Wikidata, we augment the model and incorporate knowledge graph information into the similarity measure. We compare the machine-based predicted similarity with the human-based semantic similarity. |
| Keywords | deep learning, knowledge graph, Wikidata, semantic similarity, visual similarity, visual semantic similarity |
| Type | Conference paper [With referee] |
| Conference | DL4KGS |
| Year | 2018 Month April |
| Publisher | Department of Applied Mathmatics and Computer Science, Technical University of Denmark |
| Address | Building 321, DK-2800 Kgs. Lyngby |
| Electronic version(s) | [pdf] |
| BibTeX data | [bibtex] |
| IMM Group(s) | Intelligent Signal Processing |