tanner/obsidian-tannersearch
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'master' into feature/32-suggestions

Browse Source
This commit is contained in:
Simon Cambier
2022-06-19 20:33:06 +02:00
parent 9a80e71235 a923b68b88
commit 22576acf55
27 changed files with 1391 additions and 754 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
CHANGELOG.md
View File

@@ -1,11 +1,33 @@
# Omnisearch Changelog
## 1.2.0, 1.2.1
## 1.3.x
### New
* Chinese support by @aidenlx in https://github.com/scambier/obsidian-omnisearch/pull/37
* You need to install https://github.com/aidenlx/cm-chs-patch to enable this feature
* Settings page https://github.com/scambier/obsidian-omnisearch/issues/41
* Do not show indexing Notice by default by @chrisgrieser in https://github.com/scambier/obsidian-omnisearch/pull/46
* Include notes that don't exist https://github.com/scambier/obsidian-omnisearch/issues/14
### Improved
* Better accessibility https://github.com/scambier/obsidian-omnisearch/issues/50
* Note aliases are now scored as high as the filename in search results https://github.com/scambier/obsidian-omnisearch/issues/34
* By default, reindexing is now done when the app is out of focus, and not after each save https://github.com/scambier/obsidian-omnisearch/issues/57
* On mobile, indexing is only done at startup
### Fixed
* Showing an error when a note can't be created https://github.com/scambier/obsidian-omnisearch/issues/52
## 1.2.x
### New
* #42 Files that are present in Obsidian's "Excluded Files" list are downranked by a factor of 3 (_desktop only_)
## 1.2.1
## 1.1.1
### Fixes
* Fixed a crash when no results were returned
@@ -28,4 +50,4 @@ This works as a "post-search" filter and does not allow for partial words search
## 1.0.0
* First non-beta release
* Includes Vault search and In-File search
* Includes Vault search and In-File search

9
CONTRIBUTING.md
View File

@@ -13,6 +13,12 @@ Please read this document before beginning work on a Pull Request.
- Omnisearch is still in its infancy: some important features are missing, and there will be architectural changes.
- As such, I may refuse your PR simply because it will have to be refactored in a short-ish term
## "Good First Issue"
Are you a beginner, looking for a small open source contribution? Look at the "[good first issues](https://github.com/scambier/obsidian-omnisearch/labels/good%20first%20issue)". Those issues have a limited scope, don't require intricate knowledge of the code, and are easy enough to locate, fix, and test.
If you wish to work on one of these issues, leave a comment and I'll assign it to you and give you some pointers.
## Code guidelines
- Respect the existing style
@@ -38,4 +44,5 @@ Always respect those UI & UX points:
## Style guidelines
(todo)
- .ts files must be formatted with "Prettier ESLint"
- .svelte files must be formatted with "Svelte for VS Code"

18
README.md
View File

@@ -1,9 +1,11 @@
# Omnisearch for Obsidian
[![Sponsor me](https://img.shields.io/badge/%E2%9D%A4%20Like%20this%20plugin%3F-Sponsor%20me!-ff69b4)](https://github.com/sponsors/scambier)
![GitHub release (latest by date including pre-releases)](https://img.shields.io/github/v/release/scambier/obsidian-omnisearch) ![GitHub release (latest by date including pre-releases)](https://img.shields.io/github/v/release/scambier/obsidian-omnisearch?include_prereleases&label=BRAT%20beta) [![Active Development](https://img.shields.io/badge/Maintenance%20Level-Actively%20Developed-brightgreen.svg)](https://gist.github.com/cheerfulstoic/d107229326a01ff0f333a1d3476e068d)
**Omnisearch** is a search engine that "_just works_". Type what you're looking for, and it will instantly show you the most relevant results.
**Omnisearch** is a search engine that "_just works_". It always instantly shows you the most relevant results, thanks to its smart weighting algorithm.
Under the hood, it uses the excellent [MiniSearch](https://github.com/lucaong/minisearch) library.
@@ -11,10 +13,10 @@ Under the hood, it uses the excellent [MiniSearch](https://github.com/lucaong/mi
## Features
- Keyboard-centric, you never have to use your mouse
- Automatic document scoring using the [BM25 algorithm](https://github.com/lucaong/minisearch/issues/129#issuecomment-1046257399)
- The relevance of a document against a query depends on the number of times the query terms appear in the document, its filename, and its headings
- Instant search results, with highlighting
- Keyboard first: you never have to use your mouse
- Instant & highlighted search results
- Resistance to typos
- In-file search to quickly skim multiple results in a single note
- Search filters: expressions in quotes and exclusions
@@ -22,8 +24,8 @@ Under the hood, it uses the excellent [MiniSearch](https://github.com/lucaong/mi
## Installation
- Omnisearch is available on [the official Community Plugins repository](https://obsidian.md/plugins?search=omnisearch#).
- You can also install it through [BRAT](https://github.com/TfTHacker/obsidian42-brat) for pre-releases. Be advised that those versions can be buggy.
- Omnisearch is available on [the official Community Plugins repository](https://obsidian.md/plugins?search=Omnisearch).
- Beta releases can be installed through [BRAT](https://github.com/TfTHacker/obsidian42-brat). **Be advised that those versions can be buggy.**
You can check the [CHANGELOG](./CHANGELOG.md) for more information on the different versions.
@@ -33,13 +35,13 @@ Omnisearch can be used within 2 different contexts:
### Vault Search
Omnisearch's core feature, accessible with the Command Palette "_Omnisearch: Vault search_". This modal searches through your vault and returns the most relevant notes first. The notes that contain the query terms in their filename or headings are weighted higher than the others.
Omnisearch's core feature, accessible with the Command Palette "**_Omnisearch: Vault search_**". This modal searches through your vault and returns the most relevant notes. That's all you need to _find_ a note.
If you need to list all the matches of a single note, you can do so by using `alt+enter` to open the In-File Search.
If you want to list all the search matches of a single note, you can do so by using `alt+enter` to open the In-File Search.
### In-File Search
Also accessible through the command palette "_Omnisearch: In-file search_". This modal searches through the active note's content and lists the results.
Also accessible through the Command Palette "**_Omnisearch: In-file search_**". This modal searches through the active note's content and lists the matching results. Just press enter to automatically scroll to the right place.
## Customization

2
manifest-beta.json
View File

@@ -1,7 +1,7 @@
{
"id": "omnisearch",
"name": "Omnisearch",
"version": "1.2.1",
"version": "1.3.5-beta3",
"minAppVersion": "0.14.2",
"description": "A search engine that just works",
"author": "Simon Cambier",

2
manifest.json
View File

@@ -1,7 +1,7 @@
{
"id": "omnisearch",
"name": "Omnisearch",
"version": "1.2.1",
"version": "1.3.4",
"minAppVersion": "0.14.2",
"description": "A search engine that just works",
"author": "Simon Cambier",

16
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "scambier.obsidian-search",
"version": "1.2.1",
"version": "1.3.5-beta3",
"description": "A search engine for Obsidian",
"main": "dist/main.js",
"scripts": {
@@ -15,14 +15,14 @@
"author": "Simon Cambier",
"license": "GPL-3",
"devDependencies": {
"@babel/preset-env": "^7.16.11",
"@babel/preset-env": "^7.17.10",
"@babel/preset-typescript": "^7.16.7",
"@testing-library/jest-dom": "^5.16.4",
"@tsconfig/svelte": "^3.0.0",
"@types/jest": "^27.4.1",
"@types/node": "^16.11.27",
"@typescript-eslint/eslint-plugin": "^5.20.0",
"@typescript-eslint/parser": "^5.20.0",
"@types/jest": "^27.5.0",
"@types/node": "^16.11.34",
"@typescript-eslint/eslint-plugin": "^5.23.0",
"@typescript-eslint/parser": "^5.23.0",
"babel-jest": "^27.5.1",
"builtin-modules": "^3.2.0",
"esbuild": "0.13.12",
@@ -38,11 +38,11 @@
"obsidian": "latest",
"prettier": "^2.6.2",
"prettier-eslint": "^13.0.0",
"svelte": "^3.47.0",
"svelte": "^3.48.0",
"svelte-jester": "^2.3.2",
"svelte-preprocess": "^4.10.6",
"tslib": "2.3.1",
"typescript": "^4.6.3"
"typescript": "^4.6.4"
},
"dependencies": {
"minisearch": "^5.0.0-beta1"

633
pnpm-lock.yaml generated
View File

File diff suppressed because it is too large Load Diff

8
src/__tests__/query-tests.ts
View File

@@ -40,6 +40,14 @@ describe('The Query class', () => {
).toBeTruthy()
})
it('should not exclude words when there is no space before', () => {
// Act
const query = new Query('foo bar-baz')
// Assert
expect(query.exclusions).toHaveLength(0)
})
describe('.getExactTerms()', () => {
it('should an array of strings containg "exact" values', () => {
// Act

49
src/__tests__/utils-tests.ts Normal file
View File

@@ -0,0 +1,49 @@
import type { CachedMetadata } from 'obsidian'
import { getAliasesFromMetadata } from '../utils'
describe('Utils', () => {
describe('getAliasesFromMetadata', () => {
it('should return an empty array if no metadata is provided', () => {
// Act
const actual = getAliasesFromMetadata(null)
// Assert
expect(actual).toEqual([])
})
it('should return an empty array if no aliases are provided', () => {
// Act
const actual = getAliasesFromMetadata({})
// Assert
expect(actual).toEqual([])
})
it('should return the aliases array as-is', () => {
// Arrange
const metadata = {
frontmatter: { aliases: ['foo', 'bar'] },
} as CachedMetadata
// Act
const actual = getAliasesFromMetadata(metadata)
// Assert
expect(actual).toEqual(['foo', 'bar'])
})
it('should convert the aliases string into an array', () => {
// Arrange
const metadata = {
frontmatter: { aliases: 'foo, bar' },
} as CachedMetadata
// Act
const actual = getAliasesFromMetadata(metadata)
// Assert
expect(actual).toEqual(['foo', 'bar'])
})
it('should return an empty array if the aliases field is an empty string', () => {
// Arrange
const metadata = {
frontmatter: { aliases: '' },
} as CachedMetadata
// Act
const actual = getAliasesFromMetadata(metadata)
// Assert
expect(actual).toEqual([])
})
})
})

2
src/components/GlyphAddNote.svelte Normal file
View File

@@ -0,0 +1,2 @@
<script lang="ts"></script>
<span class="suggestion-flair" aria-label="Not created yet, select to create"><svg viewBox="0 0 100 100" class="add-note-glyph" width="16" height="16"><path fill="currentColor" stroke="currentColor" d="M23.3,6.7c-3.7,0-6.7,3-6.7,6.7v73.3c0,3.7,3,6.7,6.7,6.7h28.4c-3.2-4.8-5.1-10.5-5.1-16.7c0-16.6,13.4-30,30-30 c2.3,0,4.5,0.3,6.7,0.8V31.7c0-0.9-0.3-1.7-1-2.4L60.7,7.6c-0.6-0.6-1.5-1-2.4-1L23.3,6.7z M56.7,13L77,33.3H60 c-1.8,0-3.3-1.5-3.3-3.3L56.7,13z M76.7,53.3c-12.9,0-23.3,10.4-23.3,23.3S63.8,100,76.7,100S100,89.6,100,76.7 S89.6,53.3,76.7,53.3z M76.7,63.3c1.8,0,3.3,1.5,3.3,3.3v6.7h6.7c1.8,0,3.3,1.5,3.3,3.3c0,1.8-1.5,3.3-3.3,3.3H80v6.7 c0,1.8-1.5,3.3-3.3,3.3c-1.8,0-3.3-1.5-3.3-3.3V80h-6.7c-1.8,0-3.3-1.5-3.3-3.3s1.5-3.3,3.3-3.3h6.7v-6.7 C73.3,64.8,74.8,63.3,76.7,63.3L76.7,63.3z"></path></svg></span>

3
src/components/InputSearch.svelte
View File

@@ -1,5 +1,6 @@
<script lang="ts">
import { debounce } from "obsidian"
import { toggleInputComposition } from "src/globals"
import { createEventDispatcher, onMount, tick } from "svelte"
export let value = ""
@@ -22,6 +23,8 @@ const debouncedOnInput = debounce(() => {
bind:value
bind:this={elInput}
on:input={debouncedOnInput}
on:compositionstart={(_) => toggleInputComposition(true)}
on:compositionend={(_) => toggleInputComposition(false)}
type="text"
class="prompt-input"
placeholder="Type to search through your notes"

2
src/components/ModalInFile.svelte
View File

@@ -71,11 +71,13 @@ $: {
function getGroups(matches: SearchMatch[]): SearchMatch[][] {
const groups: SearchMatch[][] = []
let lastOffset = -1
let count = 0 // TODO: FIXME: this is a hack to avoid infinite loops
while (true) {
const group = getGroupedMatches(matches, lastOffset, excerptAfter)
if (!group.length) break
lastOffset = group.last()!.offset
groups.push(group)
if (++count > 100) break
}
return groups
}

13
src/components/ModalVault.svelte
View File

@@ -3,13 +3,13 @@ let lastSearch = ""
</script>
<script lang="ts">
import { TFile } from "obsidian"
import { Notice, TFile } from "obsidian"
import { onMount, tick } from "svelte"
import InputSearch from "./InputSearch.svelte"
import ModalContainer from "./ModalContainer.svelte"
import { eventBus, type ResultNote } from "src/globals"
import { createNote, openNote } from "src/notes"
import { getSuggestions } from "src/search"
import { getSuggestions, reindexNotes } from "src/search"
import { loopIndex } from "src/utils"
import { OmnisearchInFileModal, type OmnisearchVaultModal } from "src/modals"
import ResultItemVault from "./ResultItemVault.svelte"
@@ -29,6 +29,7 @@ $: if (searchQuery) {
}
onMount(() => {
reindexNotes()
searchQuery = lastSearch
eventBus.on("vault", "enter", onInputEnter)
eventBus.on("vault", "shift-enter", onInputShiftEnter)
@@ -67,7 +68,13 @@ function onInputCtrlEnter(): void {
}
async function onInputShiftEnter(): Promise<void> {
await createNote(searchQuery)
try {
await createNote(searchQuery)
}
catch(e) {
new Notice((e as Error).message)
return
}
modal.close()
}

6
src/components/ResultItemContainer.svelte
View File

@@ -1,8 +1,9 @@
<script lang="ts">
import { createEventDispatcher } from "svelte"
import GlyphAddNote from "./GlyphAddNote.svelte"
export let id: string
export let selected = false
export let glyph = false
</script>
<div
@@ -13,5 +14,8 @@ export let selected = false
on:click
on:auxclick
>
{#if glyph}
<GlyphAddNote />
{/if}
<slot />
</div>

16
src/components/ResultItemVault.svelte
View File

@@ -1,4 +1,6 @@
<script lang="ts">
import { getNoteFromCache } from "src/notes"
import { settings } from "src/settings"
import type { ResultNote } from "../globals"
import { getMatches } from "../search"
import { highlighter, makeExcerpt, stringsToRegex } from "../utils"
@@ -10,16 +12,20 @@ export let note: ResultNote
$: reg = stringsToRegex(note.foundWords)
$: matches = getMatches(note.content, reg)
$: cleanedContent = makeExcerpt(note.content, note.matches[0]?.offset ?? -1)
$: glyph = getNoteFromCache(note.path)?.doesNotExist
$: title = settings.showShortName ? note.basename : note.path
</script>
<ResultItemContainer id={note.path} {selected} on:mousemove on:click>
<ResultItemContainer id={note.path} {selected} on:mousemove on:click {glyph}>
<span class="omnisearch-result__title">
{@html note.basename.replace(reg, highlighter)}
{@html title.replace(reg, highlighter)}
</span>
<span class="omnisearch-result__counter">
{matches.length}&nbsp;{matches.length > 1 ? "matches" : "match"}
</span>
{#if matches.length > 0}
<span class="omnisearch-result__counter">
{matches.length}&nbsp;{matches.length > 1 ? "matches" : "match"}
</span>
{/if}
<div class="omnisearch-result__body">
{@html cleanedContent.replace(reg, highlighter)}
</div>

15
src/globals.ts
View File

@@ -23,10 +23,17 @@ export type SearchNote = {
export type IndexedNote = {
path: string
basename: string
mtime: number
content: string
aliases: string
tags: string[],
headings1: string
headings2: string
headings3: string
doesNotExist?: boolean
parent?: string
}
export type SearchMatch = {
@@ -46,5 +53,13 @@ export type ResultNote = {
matches: SearchMatch[]
}
let inComposition = false
export function toggleInputComposition(toggle: boolean): void {
inComposition = toggle
}
export function isInputComposition(): boolean {
return inComposition
}
export const SPACE_OR_PUNCTUATION =
/[|\n\r -#%-*,-/:;?@[-\]_{}\u00A0\u00A1\u00A7\u00AB\u00B6\u00B7\u00BB\u00BF\u037E\u0387\u055A-\u055F\u0589\u058A\u05BE\u05C0\u05C3\u05C6\u05F3\u05F4\u0609\u060A\u060C\u060D\u061B\u061E\u061F\u066A-\u066D\u06D4\u0700-\u070D\u07F7-\u07F9\u0830-\u083E\u085E\u0964\u0965\u0970\u09FD\u0A76\u0AF0\u0C77\u0C84\u0DF4\u0E4F\u0E5A\u0E5B\u0F04-\u0F12\u0F14\u0F3A-\u0F3D\u0F85\u0FD0-\u0FD4\u0FD9\u0FDA\u104A-\u104F\u10FB\u1360-\u1368\u1400\u166E\u1680\u169B\u169C\u16EB-\u16ED\u1735\u1736\u17D4-\u17D6\u17D8-\u17DA\u1800-\u180A\u1944\u1945\u1A1E\u1A1F\u1AA0-\u1AA6\u1AA8-\u1AAD\u1B5A-\u1B60\u1BFC-\u1BFF\u1C3B-\u1C3F\u1C7E\u1C7F\u1CC0-\u1CC7\u1CD3\u2000-\u200A\u2010-\u2029\u202F-\u2043\u2045-\u2051\u2053-\u205F\u207D\u207E\u208D\u208E\u2308-\u230B\u2329\u232A\u2768-\u2775\u27C5\u27C6\u27E6-\u27EF\u2983-\u2998\u29D8-\u29DB\u29FC\u29FD\u2CF9-\u2CFC\u2CFE\u2CFF\u2D70\u2E00-\u2E2E\u2E30-\u2E4F\u3000-\u3003\u3008-\u3011\u3014-\u301F\u3030\u303D\u30A0\u30FB\uA4FE\uA4FF\uA60D-\uA60F\uA673\uA67E\uA6F2-\uA6F7\uA874-\uA877\uA8CE\uA8CF\uA8F8-\uA8FA\uA8FC\uA92E\uA92F\uA95F\uA9C1-\uA9CD\uA9DE\uA9DF\uAA5C-\uAA5F\uAADE\uAADF\uAAF0\uAAF1\uABEB\uFD3E\uFD3F\uFE10-\uFE19\uFE30-\uFE52\uFE54-\uFE61\uFE63\uFE68\uFE6A\uFE6B\uFF01-\uFF03\uFF05-\uFF0A\uFF0C-\uFF0F\uFF1A\uFF1B\uFF1F\uFF20\uFF3B-\uFF3D\uFF3F\uFF5B\uFF5D\uFF5F-\uFF65]+/u

19
src/main.ts
View File

@@ -1,14 +1,22 @@
import { Plugin, TFile } from 'obsidian'
import {
addNoteToReindex,
addToIndex,
initGlobalSearchIndex,
removeFromIndex,
removeFromIndexByPath,
} from './search'
import { OmnisearchInFileModal, OmnisearchVaultModal } from './modals'
import { loadSettings, SettingsTab } from './settings'
import { OmnisearchSuggest } from './suggestions'
// let mainWindow: { on: any; off: any } | null = null
// try {
// mainWindow = require('electron').remote.getCurrentWindow()
// }
// catch (e) {
// console.log("Can't load electron, mobile platform")
// }
export default class OmnisearchPlugin extends Plugin {
async onload(): Promise<void> {
await loadSettings(this)
@@ -41,19 +49,18 @@ export default class OmnisearchPlugin extends Plugin {
)
this.registerEvent(
this.app.vault.on('delete', file => {
removeFromIndex(file)
removeFromIndex(file.path)
}),
)
this.registerEvent(
this.app.vault.on('modify', async file => {
removeFromIndex(file)
await addToIndex(file)
addNoteToReindex(file)
}),
)
this.registerEvent(
this.app.vault.on('rename', async (file, oldPath) => {
if (file instanceof TFile && file.path.endsWith('.md')) {
removeFromIndexByPath(oldPath)
removeFromIndex(oldPath)
await addToIndex(file)
}
}),
@@ -62,4 +69,6 @@ export default class OmnisearchPlugin extends Plugin {
await initGlobalSearchIndex()
})
}
onunload(): void {}
}

49
src/modals.ts
View File

@@ -1,7 +1,8 @@
import { App, Modal, TFile } from 'obsidian'
import ModalVault from './components/ModalVault.svelte'
import ModalInFile from './components/ModalInFile.svelte'
import { eventBus } from './globals'
import { eventBus, isInputComposition } from './globals'
import { settings } from './settings'
abstract class OmnisearchModal extends Modal {
constructor(app: App) {
@@ -17,6 +18,9 @@ abstract class OmnisearchModal extends Modal {
this.modalEl.tabIndex = -1
// Setup events that can be listened through the event bus
// #region Up/Down navigation
this.scope.register([], 'ArrowDown', e => {
e.preventDefault()
eventBus.emit('arrow-down')
@@ -25,6 +29,39 @@ abstract class OmnisearchModal extends Modal {
e.preventDefault()
eventBus.emit('arrow-up')
})
// Ctrl+j/k
for (const key of [
{ k: 'j', dir: 'down' },
{ k: 'k', dir: 'up' },
] as const) {
for (const modifier of ['Ctrl', 'Meta'] as const) {
this.scope.register([modifier], key.k, e => {
if (settings.CtrlJK && this.app.vault.getConfig('vimMode')) {
e.preventDefault()
eventBus.emit('arrow-' + key.dir)
}
})
}
}
// Ctrl+n/p
for (const key of [
{ k: 'n', dir: 'down' },
{ k: 'p', dir: 'up' },
] as const) {
for (const modifier of ['Ctrl', 'Meta'] as const) {
this.scope.register([modifier], key.k, e => {
if (settings.CtrlNP && this.app.vault.getConfig('vimMode')) {
e.preventDefault()
eventBus.emit('arrow-' + key.dir)
}
})
}
}
// #endregion Up/Down navigation
this.scope.register(['Ctrl'], 'Enter', e => {
e.preventDefault()
eventBus.emit('ctrl-enter') // Open in new pane
@@ -33,17 +70,23 @@ abstract class OmnisearchModal extends Modal {
e.preventDefault()
eventBus.emit('ctrl-enter') // Open in new pane (but on Mac)
})
this.scope.register(['Alt'], 'Enter', e => {
e.preventDefault()
eventBus.emit('alt-enter') // Open the InFile modal
})
this.scope.register(['Shift'], 'Enter', e => {
e.preventDefault()
eventBus.emit('shift-enter') // Create a new note
})
this.scope.register([], 'Enter', e => {
e.preventDefault()
eventBus.emit('enter') // Open in current pane
if (!isInputComposition()) {
// Check if the user is still typing
e.preventDefault()
eventBus.emit('enter') // Open in current pane
}
})
}
}

118
src/notes.ts
View File

@@ -1,6 +1,58 @@
import { MarkdownView } from 'obsidian'
import type { ResultNote } from './globals'
import {
MarkdownView,
TFile,
WorkspaceLeaf,
type CachedMetadata,
} from 'obsidian'
import type { IndexedNote, ResultNote } from './globals'
import { stringsToRegex } from './utils'
import { settings } from './settings'
/**
* This is an in-memory cache of the notes, with all their computed fields
* used by the search engine.
* This cache allows us to quickly de-index notes when they are deleted or updated.
*/
export let notesCache: Record<string, IndexedNote> = {}
const notesCacheFilePath = `${app.vault.configDir}/plugins/omnisearch/notesCache.json`
export function resetNotesCache(): void {
notesCache = {}
}
export async function loadNotesCache(): Promise<void> {
if (
settings.storeIndexInFile &&
(await app.vault.adapter.exists(notesCacheFilePath))
) {
try {
const json = await app.vault.adapter.read(notesCacheFilePath)
notesCache = JSON.parse(json)
console.log('Notes cache loaded from the file')
}
catch (e) {
console.trace('Could not load Notes cache from the file')
console.error(e)
}
}
if (!notesCache) {
notesCache = {}
}
}
export function getNoteFromCache(key: string): IndexedNote | undefined {
return notesCache[key]
}
export function getNonExistingNotesFromCache(): IndexedNote[] {
return Object.values(notesCache).filter(note => note.doesNotExist)
}
export function addNoteToCache(filename: string, note: IndexedNote): void {
notesCache[filename] = note
}
export function removeNoteFromCache(key: string): void {
delete notesCache[key]
}
export async function openNote(
item: ResultNote,
@@ -9,7 +61,26 @@ export async function openNote(
const reg = stringsToRegex(item.foundWords)
reg.exec(item.content)
const offset = reg.lastIndex
await app.workspace.openLinkText(item.path, '', newPane)
// Check if the note is already open
// const pane = MarkdownView.getPane(item.path)
// Check if the note is already open,
// to avoid opening it twice if the first one is pinned
let existing = false
app.workspace.iterateAllLeaves(leaf => {
if (leaf.view instanceof MarkdownView) {
if (leaf.getViewState().state?.file === item.path) {
app.workspace.setActiveLeaf(leaf, false, true)
existing = true
}
}
})
if (!existing) {
// Open a new note
await app.workspace.openLinkText(item.path, '', newPane)
}
const view = app.workspace.getActiveViewOfType(MarkdownView)
if (!view) {
@@ -38,5 +109,46 @@ export async function createNote(name: string): Promise<void> {
}
catch (e) {
console.error(e)
throw e
}
}
/**
* For a given file, returns a list of links leading to notes that don't exist
* @param file
* @param metadata
* @returns
*/
export function getNonExistingNotes(
file: TFile,
metadata: CachedMetadata,
): string[] {
return (metadata.links ?? [])
.map(l => {
const path = removeAnchors(l.link)
return app.metadataCache.getFirstLinkpathDest(path, file.path)
? ''
: l.link
})
.filter(l => !!l)
}
/**
* Removes anchors and headings
* @param name
* @returns
*/
export function removeAnchors(name: string): string {
return name.split(/[\^#]+/)[0]
}
export async function saveNotesCacheToFile(): Promise<void> {
const json = JSON.stringify(notesCache)
await app.vault.adapter.write(notesCacheFilePath, json)
console.log('Notes cache saved to the file')
}
export function isCacheOutdated(file: TFile): boolean {
const indexedNote = getNoteFromCache(file.path)
return !indexedNote || indexedNote.mtime !== file.stat.mtime
}

336
src/query.ts
View File

@@ -1,4 +1,6 @@
import { stripSurroundingQuotes } from './utils'
import { settings } from './settings'
import { removeDiacritics, stripSurroundingQuotes } from './utils'
import { parseQuery } from './vendor/parse-query'
type QueryToken = {
/**
@@ -20,6 +22,7 @@ export class Query {
public exclusions: QueryToken[] = []
constructor(text = '') {
if (settings.ignoreDiacritics) text = removeDiacritics(text)
const tokens = parseQuery(text.toLowerCase(), { tokenize: true })
this.exclusions = tokens.exclude.text
.map(this.formatToken)
@@ -47,334 +50,3 @@ export class Query {
}
}
}
/*!
* search-query-parser.js
* Original: https://github.com/nepsilon/search-query-parser
* Modified by Simon Cambier
* Copyright(c) 2014-2019
* MIT Licensed
*/
interface SearchParserOptions {
offsets?: boolean
tokenize: true
keywords?: string[]
ranges?: string[]
alwaysArray?: boolean
}
interface ISearchParserDictionary {
[key: string]: any
}
type SearchParserKeyWordOffset = {
keyword: string
value?: string
}
type SearchParserTextOffset = {
text: string
}
type SearchParserOffset = (
| SearchParserKeyWordOffset
| SearchParserTextOffset
) & {
offsetStart: number
offsetEnd: number
}
interface SearchParserResult extends ISearchParserDictionary {
text: string[]
offsets: SearchParserOffset[]
exclude: { text: string[] }
}
function parseQuery(
string: string,
options: SearchParserOptions,
): SearchParserResult {
// Set a default options object when none is provided
if (!options) {
options = { offsets: true, tokenize: true }
}
else {
// If options offsets was't passed, set it to true
options.offsets =
typeof options.offsets === 'undefined' ? true : options.offsets
}
if (!string) {
string = ''
}
// Our object to store the query object
const query: SearchParserResult = {
text: [],
offsets: [],
exclude: { text: [] },
}
// When offsets is true, create their array
if (options.offsets) {
query.offsets = []
}
const exclusion: ISearchParserDictionary & { text: string[] } = { text: [] }
const terms = []
// Get a list of search terms respecting single and double quotes
const regex =
/(\S+:'(?:[^'\\]|\\.)*')|(\S+:"(?:[^"\\]|\\.)*")|(-?"(?:[^"\\]|\\.)*")|(-?'(?:[^'\\]|\\.)*')|\S+|\S+:\S+/g
let match
while ((match = regex.exec(string)) !== null) {
let term = match[0]
const sepIndex = term.indexOf(':')
// Terms that contain a `:`
if (sepIndex !== -1) {
const key = term.slice(0, sepIndex)
let val = term.slice(sepIndex + 1)
// Strip backslashes respecting escapes
val = (val + '').replace(/\\(.?)/g, function (s, n1) {
switch (n1) {
case '\\':
return '\\'
case '0':
return '\u0000'
case '':
return ''
default:
return n1
}
})
terms.push({
keyword: key,
value: val,
offsetStart: match.index,
offsetEnd: match.index + term.length,
})
}
// Other terms
else {
let isExcludedTerm = false
if (term[0] === '-') {
isExcludedTerm = true
term = term.slice(1)
}
// Strip backslashes respecting escapes
term = (term + '').replace(/\\(.?)/g, function (s, n1) {
switch (n1) {
case '\\':
return '\\'
case '0':
return '\u0000'
case '':
return ''
default:
return n1
}
})
if (isExcludedTerm) {
exclusion.text.push(term)
}
else {
terms.push({
text: term,
offsetStart: match.index,
offsetEnd: match.index + term.length,
})
}
}
}
// Reverse to ensure proper order when pop()'ing.
terms.reverse()
// For each search term
let term
while ((term = terms.pop())) {
// When just a simple term
if (term.text) {
// We add it as pure text
query.text.push(term.text)
// When offsets is true, push a new offset
if (options.offsets) {
query.offsets.push(term)
}
}
// We got an advanced search syntax
else if (term.keyword) {
let key = term.keyword
// Check if the key is a registered keyword
options.keywords = options.keywords || []
let isKeyword = false
let isExclusion = false
if (!/^-/.test(key)) {
isKeyword = !(options.keywords.indexOf(key) === -1)
}
else if (key[0] === '-') {
const _key = key.slice(1)
isKeyword = !(options.keywords.indexOf(_key) === -1)
if (isKeyword) {
key = _key
isExclusion = true
}
}
// Check if the key is a registered range
options.ranges = options.ranges || []
const isRange = !(options.ranges.indexOf(key) === -1)
// When the key matches a keyword
if (isKeyword) {
// When offsets is true, push a new offset
if (options.offsets) {
query.offsets.push({
keyword: key,
value: term.value,
offsetStart: isExclusion ? term.offsetStart + 1 : term.offsetStart,
offsetEnd: term.offsetEnd,
})
}
const value = term.value
// When value is a thing
if (value.length) {
// Get an array of values when several are there
const values = value.split(',')
if (isExclusion) {
if (exclusion[key]) {
// ...many times...
if (exclusion[key] instanceof Array) {
// ...and got several values this time...
if (values.length > 1) {
// ... concatenate both arrays.
exclusion[key] = exclusion[key].concat(values)
}
else {
// ... append the current single value.
exclusion[key].push(value)
}
}
// We saw that keyword only once before
else {
// Put both the current value and the new
// value in an array
exclusion[key] = [exclusion[key]]
exclusion[key].push(value)
}
}
// First time we see that keyword
else {
// ...and got several values this time...
if (values.length > 1) {
// ...add all values seen.
exclusion[key] = values
}
// Got only a single value this time
else {
// Record its value as a string
if (options.alwaysArray) {
// ...but we always return an array if option alwaysArray is true
exclusion[key] = [value]
}
else {
// Record its value as a string
exclusion[key] = value
}
}
}
}
else {
// If we already have seen that keyword...
if (query[key]) {
// ...many times...
if (query[key] instanceof Array) {
// ...and got several values this time...
if (values.length > 1) {
// ... concatenate both arrays.
query[key] = query[key].concat(values)
}
else {
// ... append the current single value.
query[key].push(value)
}
}
// We saw that keyword only once before
else {
// Put both the current value and the new
// value in an array
query[key] = [query[key]]
query[key].push(value)
}
}
// First time we see that keyword
else {
// ...and got several values this time...
if (values.length > 1) {
// ...add all values seen.
query[key] = values
}
// Got only a single value this time
else {
if (options.alwaysArray) {
// ...but we always return an array if option alwaysArray is true
query[key] = [value]
}
else {
// Record its value as a string
query[key] = value
}
}
}
}
}
}
// The key allows a range
else if (isRange) {
// When offsets is true, push a new offset
if (options.offsets) {
query.offsets.push(term)
}
const value = term.value
// Range are separated with a dash
const rangeValues = value.split('-')
// When both end of the range are specified
// keyword:XXXX-YYYY
query[key] = {}
if (rangeValues.length === 2) {
query[key].from = rangeValues[0]
query[key].to = rangeValues[1]
}
// When pairs of ranges are specified
// keyword:XXXX-YYYY,AAAA-BBBB
// else if (!rangeValues.length % 2) {
// }
// When only getting a single value,
// or an odd number of values
else {
query[key].from = value
}
}
else {
// We add it as pure text
const text = term.keyword + ':' + term.value
query.text.push(text)
// When offsets is true, push a new offset
if (options.offsets) {
query.offsets.push({
text: text,
offsetStart: term.offsetStart,
offsetEnd: term.offsetEnd,
})
}
}
}
}
// Return forged query object
query.exclude = exclusion
return query
}

263
src/search.ts
View File

@@ -1,5 +1,5 @@
import { Notice, TFile, type TAbstractFile } from 'obsidian'
import MiniSearch, { type SearchResult } from 'minisearch'
import { Notice, TAbstractFile, TFile } from 'obsidian'
import MiniSearch, { type Options, type SearchResult } from 'minisearch'
import {
chsRegex,
SPACE_OR_PUNCTUATION,
@@ -9,15 +9,31 @@ import {
} from './globals'
import {
extractHeadingsFromCache,
getAliasesFromMetadata,
getTagsFromMetadata,
removeDiacritics,
stringsToRegex,
stripMarkdownCharacters,
wait,
} from './utils'
import type { Query } from './query'
import { settings } from './settings'
import {
removeNoteFromCache,
getNoteFromCache,
getNonExistingNotes,
resetNotesCache,
addNoteToCache,
removeAnchors,
getNonExistingNotesFromCache,
loadNotesCache,
saveNotesCacheToFile,
isCacheOutdated,
} from './notes'
let minisearchInstance: MiniSearch<IndexedNote>
let indexedNotes: Record<string, IndexedNote> = {}
let isIndexChanged: boolean
const searchIndexFilePath = `${app.vault.configDir}/plugins/omnisearch/searchIndex.json`
const tokenize = (text: string): string[] => {
const tokens = text.split(SPACE_OR_PUNCTUATION)
@@ -36,38 +52,87 @@ const tokenize = (text: string): string[] => {
* and adds all the notes to the index
*/
export async function initGlobalSearchIndex(): Promise<void> {
indexedNotes = {}
minisearchInstance = new MiniSearch({
const options: Options<IndexedNote> = {
tokenize,
processTerm: (term: string) =>
(settings.ignoreDiacritics ? removeDiacritics(term) : term).toLowerCase(),
idField: 'path',
fields: ['basename', 'content', 'headings1', 'headings2', 'headings3'],
})
fields: [
'basename',
'aliases',
'content',
'headings1',
'headings2',
'headings3',
],
storeFields: ['tags'],
}
if (
settings.storeIndexInFile &&
(await app.vault.adapter.exists(searchIndexFilePath))
) {
try {
const json = await app.vault.adapter.read(searchIndexFilePath)
minisearchInstance = MiniSearch.loadJSON(json, options)
console.log('MiniSearch index loaded from the file')
await loadNotesCache()
}
catch (e) {
console.trace('Could not load MiniSearch index from the file')
console.error(e)
}
}
if (!minisearchInstance) {
minisearchInstance = new MiniSearch(options)
resetNotesCache()
}
// Index files that are already present
const start = new Date().getTime()
const files = app.vault.getMarkdownFiles()
const allFiles = app.vault.getMarkdownFiles()
let files
let notesSuffix
if (settings.storeIndexInFile) {
files = allFiles.filter(file => isCacheOutdated(file))
notesSuffix = 'modified notes'
}
else {
files = allFiles
notesSuffix = 'notes'
}
console.log(`Omnisearch - indexing ${files.length} ${notesSuffix}`)
// This is basically the same behavior as MiniSearch's `addAllAsync()`.
// We index files by batches of 10
if (files.length) {
console.log('Omnisearch - indexing ' + files.length + ' files')
}
for (let i = 0; i < files.length; ++i) {
if (i % 10 === 0) await wait(0)
const file = files[i]
if (file) await addToIndex(file)
if (file) {
if (getNoteFromCache(file.path)) {
removeFromIndex(file.path)
}
await addToIndex(file)
}
}
if (files.length > 0 && settings.showIndexingNotices) {
new Notice(
`Omnisearch - Indexed ${files.length} notes in ${
new Date().getTime() - start
}ms`,
)
}
if (files.length > 0) {
const message = `Omnisearch - Indexed ${files.length} ${notesSuffix} in ${
new Date().getTime() - start
}ms`
// Listen to the query input to trigger a search
// subscribeToQuery()
console.log(message)
if (settings.showIndexingNotices) {
new Notice(message)
}
await saveIndexToFile()
}
}
/**
@@ -84,6 +149,7 @@ async function search(query: Query): Promise<SearchResult[]> {
combineWith: 'AND',
boost: {
basename: settings.weightBasename,
aliases: settings.weightBasename,
headings1: settings.weightH1,
headings2: settings.weightH2,
headings3: settings.weightH3,
@@ -106,19 +172,20 @@ async function search(query: Query): Promise<SearchResult[]> {
const exactTerms = query.getExactTerms()
if (exactTerms.length) {
results = results.filter(r => {
const title = getNoteFromCache(r.id)?.path.toLowerCase() ?? ''
const content = stripMarkdownCharacters(
indexedNotes[r.id]?.content ?? '',
getNoteFromCache(r.id)?.content ?? '',
).toLowerCase()
return exactTerms.every(q => content.includes(q))
return exactTerms.every(q => content.includes(q) || title.includes(q))
})
}
// // If the search query contains exclude terms, filter out results that have them
// If the search query contains exclude terms, filter out results that have them
const exclusions = query.exclusions
if (exclusions.length) {
results = results.filter(r => {
const content = stripMarkdownCharacters(
indexedNotes[r.id]?.content ?? '',
getNoteFromCache(r.id)?.content ?? '',
).toLowerCase()
return exclusions.every(q => !content.includes(q.value))
})
@@ -135,7 +202,9 @@ async function search(query: Query): Promise<SearchResult[]> {
export function getMatches(text: string, reg: RegExp): SearchMatch[] {
let match: RegExpExecArray | null = null
const matches: SearchMatch[] = []
let count = 0 // TODO: FIXME: this is a hack to avoid infinite loops
while ((match = reg.exec(text)) !== null) {
if (++count > 100) break
const m = match[0]
if (m) matches.push({ match: m, offset: match.index })
}
@@ -166,16 +235,32 @@ export async function getSuggestions(
else results = []
}
else {
results = results.sort((a, b) => b.score - a.score).slice(0, 50)
results = results.slice(0, 50)
// Put the results with tags on top
const tags = query.segments
.filter(s => s.value.startsWith('#'))
.map(s => s.value)
for (const tag of tags) {
for (const result of results) {
if (result.tags.includes(tag)) {
result.score *= 100
}
}
}
}
// Map the raw results to get usable suggestions
const suggestions = results.map(result => {
const note = indexedNotes[result.id]
const note = getNoteFromCache(result.id)
if (!note) {
throw new Error(`Note "${result.id}" not indexed`)
}
// Remove '#' from tags, for highlighting
query.segments.forEach(s => {
s.value = s.value.replace(/^#/, '')
})
// Clean search matches that match quoted expressions,
// and inject those expressions instead
const foundWords = [
@@ -184,6 +269,7 @@ export async function getSuggestions(
),
...query.segments.filter(s => s.exact).map(s => s.value),
]
const matches = getMatches(note.content, stringsToRegex(foundWords))
const resultNote: ResultNote = {
score: result.score,
@@ -206,34 +292,56 @@ export async function addToIndex(file: TAbstractFile): Promise<void> {
if (!(file instanceof TFile) || file.extension !== 'md') {
return
}
// Check if the file was already indexed as non-existent,
// and if so, remove it from the index (before adding it again)
if (getNoteFromCache(file.path)?.doesNotExist) {
removeFromIndex(file.path)
}
try {
// console.log(`Omnisearch - adding ${file.path} to index`)
const fileCache = app.metadataCache.getFileCache(file)
if (indexedNotes[file.path]) {
// Look for links that lead to non-existing files,
// and index them as well
const metadata = app.metadataCache.getFileCache(file)
if (metadata) {
const nonExisting = getNonExistingNotes(file, metadata)
for (const name of nonExisting.filter(o => !getNoteFromCache(o))) {
addNonExistingToIndex(name, file.path)
}
}
if (getNoteFromCache(file.path)) {
throw new Error(`${file.basename} is already indexed`)
}
// Fetch content from the cache to index it as-is
const content = await app.vault.cachedRead(file)
const content = removeDiacritics(await app.vault.cachedRead(file))
// Make the document and index it
const note: IndexedNote = {
basename: file.basename,
content,
path: file.path,
headings1: fileCache
? extractHeadingsFromCache(fileCache, 1).join(' ')
mtime: file.stat.mtime,
tags: getTagsFromMetadata(metadata),
aliases: getAliasesFromMetadata(metadata).join(''),
headings1: metadata
? extractHeadingsFromCache(metadata, 1).join(' ')
: '',
headings2: fileCache
? extractHeadingsFromCache(fileCache, 2).join(' ')
headings2: metadata
? extractHeadingsFromCache(metadata, 2).join(' ')
: '',
headings3: fileCache
? extractHeadingsFromCache(fileCache, 3).join(' ')
headings3: metadata
? extractHeadingsFromCache(metadata, 3).join(' ')
: '',
}
minisearchInstance.add(note)
indexedNotes[note.path] = note
isIndexChanged = true
addNoteToCache(note.path, note)
}
catch (e) {
console.trace('Error while indexing ' + file.basename)
@@ -242,25 +350,84 @@ export async function addToIndex(file: TAbstractFile): Promise<void> {
}
/**
* Removes a file from the index
* @param file
* @returns
* Index a non-existing note.
* Useful to find internal links that lead (yet) to nowhere
* @param name
*/
export function removeFromIndex(file: TAbstractFile): void {
if (file instanceof TFile && file.path.endsWith('.md')) {
// console.log(`Omnisearch - removing ${file.path} from index`)
return removeFromIndexByPath(file.path)
}
export function addNonExistingToIndex(name: string, parent: string): void {
name = removeAnchors(name)
const filename = name + (name.endsWith('.md') ? '' : '.md')
if (getNoteFromCache(filename)) return
const note = {
path: filename,
basename: name,
mtime: 0,
content: '',
aliases: '',
headings1: '',
headings2: '',
headings3: '',
doesNotExist: true,
parent,
} as IndexedNote
minisearchInstance.add(note)
isIndexChanged = true
addNoteToCache(filename, note)
}
/**
* Removes a file from the index, by its path
* @param path
*/
export function removeFromIndexByPath(path: string): void {
const note = indexedNotes[path]
export function removeFromIndex(path: string): void {
if (!path.endsWith('.md')) {
console.info(`"${path}" is not a .md file`)
return
}
const note = getNoteFromCache(path)
if (note) {
minisearchInstance.remove(note)
delete indexedNotes[path]
isIndexChanged = true
removeNoteFromCache(path)
getNonExistingNotesFromCache()
.filter(n => n.parent === path)
.forEach(n => {
removeFromIndex(n.path)
})
}
else {
console.warn(`not not found under path ${path}`)
}
}
const notesToReindex = new Set<TAbstractFile>()
export function addNoteToReindex(note: TAbstractFile): void {
notesToReindex.add(note)
}
export async function reindexNotes(): Promise<void> {
if (settings.showIndexingNotices && notesToReindex.size > 0) {
new Notice(`Omnisearch - Reindexing ${notesToReindex.size} notes`, 2000)
}
for (const note of notesToReindex) {
removeFromIndex(note.path)
await addToIndex(note)
await wait(0)
}
notesToReindex.clear()
await saveIndexToFile()
}
async function saveIndexToFile(): Promise<void> {
if (settings.storeIndexInFile && minisearchInstance && isIndexChanged) {
const json = JSON.stringify(minisearchInstance)
await app.vault.adapter.write(searchIndexFilePath, json)
console.log('Omnisearch - Index saved on disk')
await saveNotesCacheToFile()
isIndexChanged = false
}
}

130
src/settings.ts
View File

@@ -9,8 +9,13 @@ interface WeightingSettings {
}
export interface OmnisearchSettings extends WeightingSettings {
showIndexingNotices: boolean
respectExcluded: boolean
ignoreDiacritics: boolean
showIndexingNotices: boolean
showShortName: boolean
CtrlJK: boolean
CtrlNP: boolean
storeIndexInFile: boolean
}
export class SettingsTab extends PluginSettingTab {
@@ -25,21 +30,12 @@ export class SettingsTab extends PluginSettingTab {
const { containerEl } = this
containerEl.empty()
// Title
const title = document.createElement('h2')
title.textContent = 'Omnisearch settings'
containerEl.appendChild(title)
// Settings main title
containerEl.createEl('h2', { text: 'Omnisearch settings' })
// Show notices
new Setting(containerEl)
.setName('Show indexing notices')
.setDesc('Show a notice when indexing is done, usually at startup.')
.addToggle(toggle =>
toggle.setValue(settings.showIndexingNotices).onChange(async v => {
settings.showIndexingNotices = v
await saveSettings(this.plugin)
}),
)
// #region Behavior
new Setting(containerEl).setName('Behavior').setHeading()
// Respect excluded files
new Setting(containerEl)
@@ -54,10 +50,71 @@ export class SettingsTab extends PluginSettingTab {
}),
)
// Ignore diacritics
new Setting(containerEl)
.setName('Ignore diacritics')
.setDesc(
'EXPERIMENTAL - Normalize diacritics in search terms. Words like "brûlée" or "žluťoučký" will be indexed as "brulee" and "zlutoucky". Needs a restart to take effect.',
)
.addToggle(toggle =>
toggle.setValue(settings.ignoreDiacritics).onChange(async v => {
settings.ignoreDiacritics = v
await saveSettings(this.plugin)
}),
)
new Setting(containerEl)
.setName('Store index in file')
.setDesc(
'EXPERIMENTAL - index is store on disk, instead of being rebuilt on every startup.',
)
.addToggle(toggle =>
toggle.setValue(settings.storeIndexInFile).onChange(async v => {
settings.storeIndexInFile = v
await saveSettings(this.plugin)
}),
)
// #endregion Behavior
// #region User Interface
new Setting(containerEl).setName('User Interface').setHeading()
// Show notices
new Setting(containerEl)
.setName('Show indexing notices')
.setDesc('Show a notice when indexing is done, usually at startup.')
.addToggle(toggle =>
toggle.setValue(settings.showIndexingNotices).onChange(async v => {
settings.showIndexingNotices = v
await saveSettings(this.plugin)
}),
)
// Display note names without the full path
new Setting(containerEl)
.setName('Hide full path in results list')
.setDesc(
'In the search results, only show the note name, without the full path.',
)
.addToggle(toggle =>
toggle.setValue(settings.showShortName).onChange(async v => {
settings.showShortName = v
await saveSettings(this.plugin)
}),
)
// #endregion User Interface
// #region Results Weighting
new Setting(containerEl).setName('Results weighting').setHeading()
new Setting(containerEl)
.setName(`File name (default: ${DEFAULT_SETTINGS.weightBasename})`)
.setName(
`File name & declared aliases (default: ${DEFAULT_SETTINGS.weightBasename})`,
)
.addSlider(cb => this.weightSlider(cb, 'weightBasename'))
new Setting(containerEl)
@@ -71,6 +128,36 @@ export class SettingsTab extends PluginSettingTab {
new Setting(containerEl)
.setName(`Headings level 3 (default: ${DEFAULT_SETTINGS.weightH3})`)
.addSlider(cb => this.weightSlider(cb, 'weightH3'))
// #endregion Results Weighting
// #region Shortcuts
new Setting(containerEl).setName('Shortcuts').setHeading()
new Setting(containerEl)
.setName(
'Use [Ctrl/Cmd]+j/k to navigate up/down in the results, if Vim mode is enabled',
)
.addToggle(toggle =>
toggle.setValue(settings.CtrlJK).onChange(async v => {
settings.CtrlJK = v
await saveSettings(this.plugin)
}),
)
new Setting(containerEl)
.setName(
'Use [Ctrl/Cmd]+n/p to navigate up/down in the results, if Vim mode is enabled',
)
.addToggle(toggle =>
toggle.setValue(settings.CtrlNP).onChange(async v => {
settings.CtrlNP = v
await saveSettings(this.plugin)
}),
)
// #endregion Shortcuts
}
weightSlider(cb: SliderComponent, key: keyof WeightingSettings): void {
@@ -85,12 +172,21 @@ export class SettingsTab extends PluginSettingTab {
}
export const DEFAULT_SETTINGS: OmnisearchSettings = {
showIndexingNotices: true,
respectExcluded: true,
ignoreDiacritics: false,
showIndexingNotices: false,
showShortName: false,
weightBasename: 2,
weightH1: 1.5,
weightH2: 1.3,
weightH3: 1.1,
CtrlJK: false,
CtrlNP: false,
storeIndexInFile: false,
} as const
export let settings: OmnisearchSettings = Object.assign({}, DEFAULT_SETTINGS)

17
src/types.d.ts vendored
View File

@@ -1,7 +1,22 @@
import { type MetadataCache } from 'obsidian'
import type { MetadataCache, ViewState, Vault } from 'obsidian'
declare module 'obsidian' {
interface MetadataCache {
isUserIgnored?(path: string): boolean
}
interface FrontMatterCache {
aliases?: string[] | string
tags?: string[] | string
}
interface ViewState {
state?: {
file?: string
}
}
interface Vault {
getConfig(string): unknown
}
}

43
src/utils.ts
View File

@@ -61,19 +61,6 @@ export function stringsToRegex(strings: string[]): RegExp {
return new RegExp(strings.map(s => `(${escapeRegex(s)})`).join('|'), 'gi')
}
export function replaceAll(
text: string,
terms: string[],
cb: (t: string) => string,
): string {
terms.sort((a, b) => a.length - b.length)
const regs = terms.map(term => new RegExp(escapeRegex(term), 'gi'))
for (const reg of regs) {
text = text.replaceAll(reg, cb)
}
return text
}
export function extractHeadingsFromCache(
cache: CachedMetadata,
level: number,
@@ -147,3 +134,33 @@ export async function filterAsync<T>(
export function stripMarkdownCharacters(text: string): string {
return text.replace(/(\*|_)+(.+?)(\*|_)+/g, (match, p1, p2) => p2)
}
export function getAliasesFromMetadata(
metadata: CachedMetadata | null,
): string[] {
const arrOrString = metadata?.frontmatter?.aliases ?? []
return (Array.isArray(arrOrString) ? arrOrString : arrOrString.split(','))
.map(s => (s ? s.trim() : s))
.filter(s => !!s)
}
export function getTagsFromMetadata(metadata: CachedMetadata | null): string[] {
const arrOrString = metadata?.frontmatter?.tags ?? []
const fromFrontMatter = (
Array.isArray(arrOrString) ? arrOrString : arrOrString.split(',')
)
.map(s => (s ? s.trim() : s))
.filter(s => !!s)
const fromBody = (metadata?.tags ?? []).map(t => t.tag)
return [...fromFrontMatter, ...fromBody].map(t =>
t[0] !== '#' ? '#' + t : t,
)
}
/**
* https://stackoverflow.com/a/37511463
*/
export function removeDiacritics(str: string): string {
return str.normalize('NFD').replace(/\p{Diacritic}/gu, '')
}

332
src/vendor/parse-query.ts vendored Normal file
View File

@@ -0,0 +1,332 @@
/*!
* search-query-parser.js
* Original: https://github.com/nepsilon/search-query-parser
* Modified by Simon Cambier
* Copyright(c) 2014-2019
* MIT Licensed
*/
interface SearchParserOptions {
offsets?: boolean
tokenize: true
keywords?: string[]
ranges?: string[]
alwaysArray?: boolean
}
interface ISearchParserDictionary {
[key: string]: any
}
type SearchParserKeyWordOffset = {
keyword: string
value?: string
}
type SearchParserTextOffset = {
text: string
}
type SearchParserOffset = (
| SearchParserKeyWordOffset
| SearchParserTextOffset
) & {
offsetStart: number
offsetEnd: number
}
interface SearchParserResult extends ISearchParserDictionary {
text: string[]
offsets: SearchParserOffset[]
exclude: { text: string[] }
}
export function parseQuery(
string: string,
options: SearchParserOptions,
): SearchParserResult {
// Set a default options object when none is provided
if (!options) {
options = { offsets: true, tokenize: true }
}
else {
// If options offsets was't passed, set it to true
options.offsets =
typeof options.offsets === 'undefined' ? true : options.offsets
}
if (!string) {
string = ''
}
// Our object to store the query object
const query: SearchParserResult = {
text: [],
offsets: [],
exclude: { text: [] },
}
// When offsets is true, create their array
if (options.offsets) {
query.offsets = []
}
const exclusion: ISearchParserDictionary & { text: string[] } = { text: [] }
const terms = []
// Get a list of search terms respecting single and double quotes
const regex =
/(\S+:'(?:[^'\\]|\\.)*')|(\S+:"(?:[^"\\]|\\.)*")|(-?"(?:[^"\\]|\\.)*")|(-?'(?:[^'\\]|\\.)*')|\S+|\S+:\S+/g
let match
let count = 0 // TODO: FIXME: this is a hack to avoid infinite loops
while ((match = regex.exec(string)) !== null) {
if (++count > 100) break
let term = match[0]
const sepIndex = term.indexOf(':')
// Terms that contain a `:`
if (sepIndex !== -1) {
const key = term.slice(0, sepIndex)
let val = term.slice(sepIndex + 1)
// Strip backslashes respecting escapes
val = (val + '').replace(/\\(.?)/g, function (s, n1) {
switch (n1) {
case '\\':
return '\\'
case '0':
return '\u0000'
case '':
return ''
default:
return n1
}
})
terms.push({
keyword: key,
value: val,
offsetStart: match.index,
offsetEnd: match.index + term.length,
})
}
// Other terms
else {
let isExcludedTerm = false
if (term[0] === '-') {
isExcludedTerm = true
term = term.slice(1)
}
// Strip backslashes respecting escapes
term = (term + '').replace(/\\(.?)/g, function (s, n1) {
switch (n1) {
case '\\':
return '\\'
case '0':
return '\u0000'
case '':
return ''
default:
return n1
}
})
if (isExcludedTerm) {
exclusion.text.push(term)
}
else {
terms.push({
text: term,
offsetStart: match.index,
offsetEnd: match.index + term.length,
})
}
}
}
// Reverse to ensure proper order when pop()'ing.
terms.reverse()
// For each search term
let term
while ((term = terms.pop())) {
// When just a simple term
if (term.text) {
// We add it as pure text
query.text.push(term.text)
// When offsets is true, push a new offset
if (options.offsets) {
query.offsets.push(term)
}
}
// We got an advanced search syntax
else if (term.keyword) {
let key = term.keyword
// Check if the key is a registered keyword
options.keywords = options.keywords || []
let isKeyword = false
let isExclusion = false
if (!/^-/.test(key)) {
isKeyword = !(options.keywords.indexOf(key) === -1)
}
else if (key[0] === '-') {
const _key = key.slice(1)
isKeyword = !(options.keywords.indexOf(_key) === -1)
if (isKeyword) {
key = _key
isExclusion = true
}
}
// Check if the key is a registered range
options.ranges = options.ranges || []
const isRange = !(options.ranges.indexOf(key) === -1)
// When the key matches a keyword
if (isKeyword) {
// When offsets is true, push a new offset
if (options.offsets) {
query.offsets.push({
keyword: key,
value: term.value,
offsetStart: isExclusion ? term.offsetStart + 1 : term.offsetStart,
offsetEnd: term.offsetEnd,
})
}
const value = term.value
// When value is a thing
if (value.length) {
// Get an array of values when several are there
const values = value.split(',')
if (isExclusion) {
if (exclusion[key]) {
// ...many times...
if (exclusion[key] instanceof Array) {
// ...and got several values this time...
if (values.length > 1) {
// ... concatenate both arrays.
exclusion[key] = exclusion[key].concat(values)
}
else {
// ... append the current single value.
exclusion[key].push(value)
}
}
// We saw that keyword only once before
else {
// Put both the current value and the new
// value in an array
exclusion[key] = [exclusion[key]]
exclusion[key].push(value)
}
}
// First time we see that keyword
else {
// ...and got several values this time...
if (values.length > 1) {
// ...add all values seen.
exclusion[key] = values
}
// Got only a single value this time
else {
// Record its value as a string
if (options.alwaysArray) {
// ...but we always return an array if option alwaysArray is true
exclusion[key] = [value]
}
else {
// Record its value as a string
exclusion[key] = value
}
}
}
}
else {
// If we already have seen that keyword...
if (query[key]) {
// ...many times...
if (query[key] instanceof Array) {
// ...and got several values this time...
if (values.length > 1) {
// ... concatenate both arrays.
query[key] = query[key].concat(values)
}
else {
// ... append the current single value.
query[key].push(value)
}
}
// We saw that keyword only once before
else {
// Put both the current value and the new
// value in an array
query[key] = [query[key]]
query[key].push(value)
}
}
// First time we see that keyword
else {
// ...and got several values this time...
if (values.length > 1) {
// ...add all values seen.
query[key] = values
}
// Got only a single value this time
else {
if (options.alwaysArray) {
// ...but we always return an array if option alwaysArray is true
query[key] = [value]
}
else {
// Record its value as a string
query[key] = value
}
}
}
}
}
}
// The key allows a range
else if (isRange) {
// When offsets is true, push a new offset
if (options.offsets) {
query.offsets.push(term)
}
const value = term.value
// Range are separated with a dash
const rangeValues = value.split('-')
// When both end of the range are specified
// keyword:XXXX-YYYY
query[key] = {}
if (rangeValues.length === 2) {
query[key].from = rangeValues[0]
query[key].to = rangeValues[1]
}
// When pairs of ranges are specified
// keyword:XXXX-YYYY,AAAA-BBBB
// else if (!rangeValues.length % 2) {
// }
// When only getting a single value,
// or an odd number of values
else {
query[key].from = value
}
}
else {
// We add it as pure text
const text = term.keyword + ':' + term.value
query.text.push(text)
// When offsets is true, push a new offset
if (options.offsets) {
query.offsets.push({
text: text,
offsetStart: term.offsetStart,
offsetEnd: term.offsetEnd,
})
}
}
}
}
// Return forged query object
query.exclude = exclusion
return query
}

5
tsconfig.json
View File

@@ -22,6 +22,7 @@
]
},
"include": [
"**/*.ts"
, "src/__tests__/event-bus-tests.mts" ]
"**/*.ts",
"src/__tests__/event-bus-tests.mts"
]
}

11
versions.json
View File

@@ -19,5 +19,14 @@
"1.1.0": "0.14.2",
"1.1.1": "0.14.2",
"1.2.0": "0.14.2",
"1.2.1": "0.14.2"
"1.2.1": "0.14.2",
"1.3.0-beta": "0.14.2",
"1.3.1-beta": "0.14.2",
"1.3.2-beta": "0.14.2",
"1.3.3-beta": "0.14.2",
"1.3.3": "0.14.2",
"1.3.4": "0.14.2",
"1.3.5-beta1": "0.14.2",
"1.3.5-beta2": "0.14.2",
"1.3.5-beta3": "0.14.2"
}