Boolean Searching

1

Is there any possibility of including or enabling (or perhaps I just don’t know the Syntax already used!) Boolean searching in Tropy? I’d specifically like to be able to search for X ANDNOT Y. Thanks!

2

This functionality is not exposed very well yet, but you can definitely use Boolean operators. The syntax is a bit quirky at the moment but you can combine phrases using AND, OR, and NOT (those must be upper-case!). Currently, Tropy will add a wildcard (*) at the end of your query – this can often lead to confusing results if you want to use more complex queries. The wildcard will not be added if there is a *, a +, or a quotation mark already in your query, so sometimes it’s helpful to add quotation marks around individual phrases, e.g. "this" OR "that" to make sure Tropy does not search for this OR that*.

The underlying mechanism used by Tropy is SQLite’s Full-text search extension; if you’re curious you can see the full query syntax here (the index consists of all the notes and metadata of your project).

Advanced search functionality is also one of the goals of our current development cycle, so stay tuned!

3

So with the quote marks I have it working—however it isn’t including the “notes” field in the search. Is this standard behavior, a setting that can be changed, or a bug?

Essentially what I want to be able to do is mark pages that are in a later hand and exclude them from searches for particular features that I’ve tagged.

4

I could add a field to the template for this info—but I’m not sure which one would be the most appropriate that I’m not already using for something else.

5

Can you show me your query and the text which does not match? Normally, all your notes should be added to the full-text index anytime the note is saved.

Regarding metadata for notes that’s actually something that came up recently and we’ll evaluate if it would make sense to allow for that. Meanwhile, you could also add a field to a photo or selection template (if your notes are attached to selections) and add the appropriate field there. That said, the notes should definitely be searchable using the quick search field, too.

6

Here is the search:

“Incipit” NOT “Supp”

Incipit is in a field [BIBO Locator] and all of these are coming up.

Supp is in the notes.

This search should exclude the one incipit that I have marked Supp but it doesn’t. If I move Supp to the same field as Incipit, it works as expected, but that would be a bit cumbersome.

Is it possible to create generic fields that aren’t tied to a specific ontology system? I’m working with Greek Mss of a rather lengthy work and there doesn’t seem to be a vocabulary that well matches what I need to track.

7

Oh, you are right: the boolean operators will not work across notes and metadata. This is because we build two individual indices for those so the query as a whole is sent to each index (i.e., you can use the boolean operations only within each index but not combining them). That’s something we’ll have to fix when adding advanced search functionality!

It’s possible to create your own ontology using random URIs; I could create such an ontology for you if you like but I would strongly advise against it, because one-off properties will make it hard, now and in the future, for other tools to process your data. What kind of fields / properties are you looking for? There are a number of metadata whizzes around here so they might be able to point you to a suitable vocabulary.

8

At present, for each item, I’m tagging the “text on image” using the BIBO location field. I’m tagging the shelf mark of the mss, and the source of the image. I’d like to add a field for whether or not the specific page is supplemental to the main mss, and some for specific layout features of the ms (e.g. number of columns, type of script etc).

9

I’m working on a new edition of a Greek Patristic work, and am attempting to use Tropy to index my mss images.

10

The DM2E model (n3 file) contains some of the properties you’re using (dm2e:incipit, dm2e:shelfmarkLocation) and others that may apply to your work (dm2e:writtenAreaDimension, dm2e:pageDimension). It looks like the model was created for purposes similar to your own, so definitely try it out.

As for a “supplemental” boolean and the layout features, I can find no vocabularies that generically provide these properties, so perhaps a one-off ontology is the way to go.

1 Like
11

So is there a way to modify a template in such a way that all text in one field is placed in a new field?

12

If all the existing values are identical it would be very easy to move them to the new field using bulk editing (basically, enable the ‘template’ column in the project view, sort by template in order to group all your items by template, select all the items using your template and, in the metadata panel cut and paste all values from the old to the new field).

Moving individual values is not currently possible using the UI (this seems very useful though, I’ll add this to our feature list!), however, it would be relatively easy to do this directly in your project database.

Here is how you can do this:

  1. Close Tropy
  2. Make a backup copy of your project file!
  3. Open your project file in SQLite3 (on macOS or Linux sqlite3 is typically installed by default)
  4. Run the following query, replacing the property and template values accordingly. This example would move all dc:type values of items using the Tropy Generic template to dcterms:type.
UPDATE metadata
  SET property = 'http://purl.org/dc/terms/type'
  WHERE property = 'http://purl.org/dc/elements/1.1/type'
    AND id IN (
      SELECT id
        FROM subjects
        WHERE template = 'https://tropy.org/v1/templates/generic'
    );

If changing the database directly sounds too dangerous, I’m happy to make the change for you if you would like to send me your project file.

13

I try to use boolean searching but it seems that works only on terms which are in the same field. For example if I search all documents who have ‘type’ is ‘letter’ and title contains ‘prunier’ I tape :

letter AND prunier

In this case, Tropy displays nothing. Do I do a mistake or it’s not possible for now ?

I try too column filters without results. Can you confirm that it should works or not actually ?

Thanks in advance.