[1.1.3] Readwise (#573)

* [1.1.3] Readwise

- Code cleanup
- Flow fixes
- BugFix: escape leading quotes in titles

Signed-off-by: Aaron Gascoigne <[email protected]>

Change highlight web handler

Signed-off-by: Aaron Gascoigne <[email protected]>

Fix some titles not being escaped

* Bugfix: Remove all new lines

Author: aaronpoweruser

aaronpoweruser.ReadwiseUnofficial/CHANGELOG.md

@@ -4,62 +4,83 @@
 
 See Plugin [README](https://github.com/NotePlan/plugins/blob/main/aaronpoweruser.ReadwiseUnofficial/README.md) for details on available commands and use case.
 
+## [1.1.3] Readwise 2024-05-22 (@aaronpoweruser)
+
+- Ignore leading escaped quotes
+- Code cleanup
+- Remove illegal characters in daily review titles.
+- Switch book highlight links to Kindle web reader as deep links are [deprecated](https://help.readwise.io/article/40-can-i-jump-to-a-highlight-directly-in-the-kindle-app).
 
 ## [1.1.2] Readwise 2024-04-22 (aaronpoweruser)
+
 - Fix Sync log dates not respecting time zones.
 
 ## [1.1.1] Readwise 2024-04-20 (aaronpoweruser)
+
 - Fix internal links in daily reviews.
 
 ## [1.1.0] Readwise 2024-04-19 (aaronpoweruser)
+
 - Refactor
 - Added a sync log see [README](https://github.com/NotePlan/plugins/blob/main/aaronpoweruser.ReadwiseUnofficial/README.md).
 - Fixed alert dialog count being wrong sometimes.
 - Fixed illegal characters and newlines in titles.
 - Add support for daily reviews via template.
 
 ## [1.0.1] Readwise 2023-08-20 (aaronpoweruser)
-* [#452] Fix pagination issue for users with large libraries thanks @TobiasMende
+
+- [#452] Fix pagination issue for users with large libraries thanks @TobiasMende
 
 ## [1.0.0] Readwise  2023-02-07 (aaronpoweruser)
-* public release
+
+- public release
 
 ## [0.1.8] Readwise  2023-02-07 (aaronpoweruser)
-* Show user notes
-* Add support for highlight headings
+
+- Show user notes
+- Add support for highlight headings
+
 ## [0.1.7] Readwise  2023-01-25 (aaronpoweruser)
-* Settings description clean up
-* Fix null highlight links
-* Fix unneeded long titles
+
+- Settings description clean up
+- Fix null highlight links
+- Fix unneeded long titles
+
 ## [0.1.6] - 2023-01-12 (aaronpoweruser)
+
 - Fix access token dialog
 - Use readable titles to make linking to notes easier
 - Add long title field
 - Allow Supplemental(readwise generated) notes to be grouped with main notes
 - Change highlights from list to quotes
 
 ## [0.1.5] - 2023-01-12 (aaronpoweruser)
+
 - Fix metadata not being created
 - Fixed double metadata when not using group by type
 - Allow empty tag prefixes
 
 Known issues:
+
 - "Don’t Set Goals… Do This Instead" causes a new note to be created
 - Using heading as metadata does not support tag updates
 - Images are not handled gracefully
 
 ## [0.1.4] - 2023-01-09 (aaronpoweruser)
+
 - Clean up new lines in notes
 - add success message
 - Fix rebuild
 
 ## [0.1.3] - 2023-01-03 (aaronpoweruser)
+
 - Add front matter support
 - Add optional tag prefix
 - Clean up settings
 - Code cleanup
 
 ## [0.1.2] - 2023-01-02 (aaronpoweruser)
+
 - Add missing highlight header
 - Increase timeout when checking for new note
 
@@ -71,6 +92,7 @@ Known issues:
 - Add link to kindle for books (fixes links being null)
 
 ## [0.1.0] - 2022-12-28 (aaronpoweruser)
+
 First release
 
 ## Changelog

aaronpoweruser.ReadwiseUnofficial/README.md

@@ -1,10 +1,11 @@
 # Readwise Unofficial Noteplan Plugin
 
-## About This Plugin 
+## About This Plugin
 
 A sync engine for readwise
 
 ### Features
+
 - Daily review's via templates
 - Full highlight download
 - Highlight updates
@@ -17,14 +18,15 @@ A sync engine for readwise
 ![General features](docs/ReadwiseSync.png)
 
 ### Todo
-* Index note
-* Image support (needs Noteplan API update)
 
-### Known issues:
+- Index note
+- Image support (needs Noteplan API update)
+
+### Known issues
+
 - Using heading as metadata does not support tag updates
 - Images are not handled gracefully
 
-
 ## Commands
 
 ### //Readwise sync
@@ -36,19 +38,17 @@ Downloads new highlights since last sync
 **Only needs to be used if highlights are deleted**
 Downloads all highlights (if they already exist they will be duplicated)
 
-
 ### //Readwise Daily review
 
 Downloads daily reviews (does not mark as read) with links to note and author.
 Can be added to a note with templates using
 
-```
+``` javascript
 <%- await DataStore.invokePluginCommandByName("Readwise Daily Review","aaronpoweruser.ReadwiseUnofficial") %>
 ```
 
 ![Daily review](docs/dailyReview.png)
 
-
 ## Settings
 
 These commands require configuration.
@@ -87,4 +87,4 @@ A note that has all highlights synced during a sync with dated references.
 
 ## Latest Updates
 
-See [CHANGELOG](https://github.com/NotePlan/plugins/blob/main/aaronpoweruser.ReadwiseUnofficial/CHANGELOG.md) for latest updates/changes to this plugin.
+See [CHANGELOG](https://github.com/NotePlan/plugins/blob/main/aaronpoweruser.ReadwiseUnofficial/CHANGELOG.md) for latest updates/changes to this plugin.

aaronpoweruser.ReadwiseUnofficial/plugin.json

@@ -4,8 +4,8 @@
   "noteplan.minAppVersion": "3.4.0",
   "plugin.id": "aaronpoweruser.ReadwiseUnofficial",
   "plugin.name": "📚 Readwise Unofficial",
-  "plugin.version": "1.1.2",
-  "plugin.lastUpdateInfo": "Fix sync log dates being wrong.\nAdd a sync log note. \nAdd support for daily reviews via templates",
+  "plugin.version": "1.1.3",
+  "plugin.lastUpdateInfo": "Ignore leading escaped quotes. Fixed links to highlights on web.\nFix sync log dates being wrong.\nAdd a sync log note. \nAdd support for daily reviews via templates",
   "plugin.description": "A sync plugin for Readwise",
   "plugin.author": "aaronpoweruser",
   "plugin.dependencies": [],
@@ -118,7 +118,7 @@
       "required": true
     },
     {
-      "COMMENT": "Enable to force resyncing of all highlights",
+      "COMMENT": "Enable to force a resync of all highlights",
       "title": "Force sync all highlights",
       "key": "forceSync",
       "type": "hidden",

aaronpoweruser.ReadwiseUnofficial/src/NPReadwise.js

@@ -1,9 +1,9 @@
 // @flow
 import { showMessage } from '../../helpers/userInput'
 import pluginJson from '../plugin.json'
-import { checkAccessToken, escapeTwitterHandle } from './NPReadwiseHelpers'
+import { checkAccessToken, escapeTwitterHandle, removeInvalidChars, removeNewlines } from './NPReadwiseHelpers'
 import { parseHighlightsAndWriteToNote } from './NPReadwiseNotes'
-import { startReadwiseSyncLog, finishReadwiseSyncLog } from './NPReadwiseSync'
+import { startReadwiseSyncLog, finishReadwiseSyncLog } from './NPReadwiseSyncLog'
 import { log, logDebug, logError } from '@helpers/dev'
 
 const LAST_SYNC_TIME = 'last_sync_time'
@@ -36,17 +36,17 @@ export async function readwiseDailyReview(): Promise<string> {
 }
 
 async function handleReadwiseSync(response: any): Promise<void> {
-  let downloadHiglightCount = 0,
+  let downloadHighlightCount = 0,
     updatedSourceCount = 0
   await startReadwiseSyncLog()
-  response.forEach((highightSource) => {
+  response.forEach((highlightSource) => {
     updatedSourceCount++
-    downloadHiglightCount += highightSource.highlights.length
-    parseHighlightsAndWriteToNote(highightSource)
+    downloadHighlightCount += highlightSource.highlights.length
+    parseHighlightsAndWriteToNote(highlightSource)
   })
-  log(pluginJson, `Downloaded ${downloadHiglightCount} highlights from Readwise. Updated ${updatedSourceCount} notes.`)
-  await showMessage(`Downloaded ${downloadHiglightCount} highlights from Readwise. Updated ${updatedSourceCount} notes.`)
-  await finishReadwiseSyncLog(downloadHiglightCount, updatedSourceCount)
+  log(pluginJson, `Downloaded ${downloadHighlightCount} highlights from Readwise. Updated ${updatedSourceCount} notes.`)
+  await showMessage(`Downloaded ${downloadHighlightCount} highlights from Readwise. Updated ${updatedSourceCount} notes.`)
+  await finishReadwiseSyncLog(downloadHighlightCount, updatedSourceCount)
 }
 
 /**
@@ -90,7 +90,6 @@ async function doReadWiseFetch(accessToken: string, lastFetchTime: string, downl
     DataStore.saveData(new Date().toISOString(), LAST_SYNC_TIME, true)
 
     const parsedJson = JSON.parse(response)
-    // DataStore.saveData(JSON.stringify(Json), 'readwise_data.json', true)
     const pageCursor = parsedJson.nextPageCursor
     logDebug(pluginJson, `page cursor is : ${pageCursor}`)
 
@@ -99,15 +98,17 @@ async function doReadWiseFetch(accessToken: string, lastFetchTime: string, downl
     if (pageCursor !== null && pageCursor !== '') {
       data = await doReadWiseFetch(accessToken, lastFetchTime, count, pageCursor)
     }
-    return parsedJson.results.concat(data)
+    const result = parsedJson.results.concat(data)
+    // DataStore.saveData(JSON.stringify(result), 'readwise_data.json', true)
+    return result
   } catch (error) {
     logError(pluginJson, error)
   }
 }
 /*
  * Gets the users Daily review from the readwise api
-  * @returns {string} - the daily review highlights
-  */
+ * @returns {string} - the daily review highlights
+ */
 async function getReadwiseDailyReview(): Promise<string> {
   const accessToken = DataStore.settings.accessToken ?? ''
   let highlightString = ''
@@ -124,15 +125,16 @@ async function getReadwiseDailyReview(): Promise<string> {
     const highlights = JSON.parse(response).highlights
 
     await highlights.map((highlight) => {
-      const formattedHighlight = `${highlight.text.replace(/\n/g, ' ')} [ [[${highlight.title}]], [[${escapeTwitterHandle(highlight.author)}]] ]`
+      const formattedHighlight = `${removeNewlines(highlight.text)} [ [[${removeInvalidChars(highlight.title)}]], [[${escapeTwitterHandle(highlight.author)}]] ]`
       highlightString += `> ${formattedHighlight}\n`
     })
-    if (highlightString.length > 1) { // remove the last newline
+    if (highlightString.endsWith('\n')) {
+      // remove the last newline
       highlightString = highlightString.substring(0, highlightString.length - 1)
     }
-    logDebug(pluginJson, `daily review highlights are \n\n ${highlightString}`)
-    return highlightString
+    logDebug(pluginJson, `Daily review highlights are\n\n ${highlightString}`)
   } catch (error) {
     logError(pluginJson, error)
   }
+  return highlightString
 }

aaronpoweruser.ReadwiseUnofficial/src/NPReadwiseHelpers.js

@@ -8,7 +8,7 @@ const READWISE_API_KEY_LENGTH = 50
 /**
  * Checks if the readwise access token is valid
  */
-export async function checkAccessToken(): void {
+export async function checkAccessToken(): Promise<void> {
   const accessToken = DataStore.settings.accessToken ?? ''
   logDebug(pluginJson, `access token is : ${accessToken}`)
 
@@ -25,38 +25,46 @@ export async function checkAccessToken(): void {
  */
 export function buildReadwiseNoteTitle(source: any): string {
   if (source.readable_title !== '') {
-    return source.readable_title
-      .replace(/\/*$/, '')
-      .replace(/^(.+)\/[^\/]*?$/, '$1')
-      .replace('\n', '')
+    return removeInvalidChars(source.readable_title)
   } else if (source.title !== '') {
-    return source.title
-      .replace(/\/*$/, '')
-      .replace(/^(.+)\/[^\/]*?$/, '$1')
-      .replace('\n', '')
+    return removeInvalidChars(source.title)
   } else {
-    return source.author
-      .replace(/\/*$/, '')
-      .replace(/^(.+)\/[^\/]*?$/, '$1')
-      .replace('\n', '')
+    return removeInvalidChars(source.author)
   }
 }
 
+/**
+ * Sanitize the string by removing invalid characters
+ * @param {string} string - the string to sanitize
+ * @returns {string} - the sanitized string
+ */
+export function removeInvalidChars(string: string): string {
+  return removeNewlines(
+    string
+      .replace(/^"/, '') // remove leading double quote
+      .trim(),
+  )
+}
+
 /**
  * Parse readwise data and generate front matter
  * @param {*} source - the readwise data as a JSON object
  * @returns
  */
 export function buildReadwiseFrontMatter(source: any): any {
   const frontMatter = {}
+  // $FlowIgnore[prop-missing] - intentionally setting properties dynamically as frontMatter keys are dynamic
   frontMatter.author = `[[${escapeTwitterHandle(source.author)}]]`
   if (source.readable_title.toLowerCase().trim() !== source.title.toLowerCase().trim()) {
-    frontMatter.long_title = source.title
+    // $FlowIgnore[prop-missing] - intentionally setting properties dynamically as frontMatter keys are dynamic
+    frontMatter.long_title = removeInvalidChars(source.title)
   }
   if (source.book_tags !== null && source.book_tags.length > 0) {
+    // $FlowIgnore[prop-missing] - intentionally setting properties dynamically as frontMatter keys are dynamic
     frontMatter.tags = source.book_tags.map((tag) => `${formatTag(tag.name)}`).join(', ')
   }
   if (source.unique_url !== null) {
+    // $FlowIgnore[prop-missing] - we are intentionally setting properties dynamically
     frontMatter.url = source.unique_url
   }
   return frontMatter
@@ -68,15 +76,15 @@ export function buildReadwiseFrontMatter(source: any): any {
  * @returns {string} - the formatted heading
  */
 export function buildReadwiseMetadataHeading(source: any): string {
-  let metadata = `author: [[${escapeTwitterHandle(source.author)}]]` + '\n'
+  let metadata = `author: [[${escapeTwitterHandle(source.author)}]]\n`
   if (source.book_tags !== null && source.book_tags.length > 0) {
     metadata += `tags: ${source.book_tags.map((tag) => `${formatTag(tag.name)}`).join(', ')}\n`
   }
   if (source.unique_url !== null) {
     metadata += `url: ${source.unique_url}`
   }
   if (source.readable_title.toLowerCase().trim() !== source.title.toLowerCase().trim()) {
-    metadata += `long_title: ${source.title}`
+    metadata += `long_title: ${removeInvalidChars(source.title)}`
   }
   return metadata
 }
@@ -95,12 +103,13 @@ function formatTag(tag: string): string {
   }
 }
 
-/*
- * removes all empty lines in a note
- * @param {Tnote} note - the note to remove empty lines from
+/**
+ * Remove all newline characters from a string
+ * @param {string} text - the text to remove newline characters from
+ * @returns {string} - the text with newline characters removed
  */
-export function removeEmptyLines(note: ?Tnote): void {
-  note.content = note?.content?.replace(/^\s*\n/gm, '')
+export function removeNewlines(text: string): string {
+  return text.replaceAll(/\n/g, ' ')
 }
 
 /**