this rootless Python script rips Windows Recall’s screenshots and SQLite database of OCRed text and allows you to search them.

  • salarua@sopuli.xyzOP
    link
    fedilink
    English
    arrow-up
    4
    ·
    edit-2
    5 months ago

    if i were designing a recall program, here’s how i would do it: it would take a screenshot every five seconds, OCR it, then run it through local quantized image recognition and word association neural networks, and then toss everything into a CryFS vault. when launching the recall program, you have to provide the password to unlock the vault so it can read and write to it. it can only run in the foreground (so you have to keep the window open for it to run, no closing it and forgetting about it) and it will display a status indicator in your system tray that provides a menu to pause or stop recording. afterwards, you can mark any text or region of the screen for redaction, and it’ll redact it across all screenshots and delete it from the database; you can delete individual screenshots or entire periods of time; and there will be an easily accessible self-destruct option that shreds the database (i.e. overwriting it with random garbage 21 times before deleting it off the disk). this is all offline and the application will not request network access

    i’m just making this up on the fly, so there are absolutely security and privacy considerations I absolutely forgot about, but this is the bare minimum i would like to see

    • csm10495@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      1
      ·
      5 months ago

      IIUC it wouldn’t be able to be automatically started then, right? I mean I guess you could drag it to startup but it would need the password to start. From a security minded perspective that’s good, but from a user perspective kind of sucks. I already unlocked the computer: as a user id just want it to ‘work’.

      There is always a tug of war between best level of security and user experience. I guess the best security is to get rid of the human element though… so eh.

      Always forced to foreground makes it even less convenient and kind of odd. I dig the status tray control though. I don’t see this functionality as being useful if you have to remember to turn it on. If I remember what I was doing enough to turn it on, I’d write down what I’d forget. To me it’s about allowing the user to pick their comfort level.

      I figure the cryptfs could be a bitlocker volume with a different key than the base C drives key to get similar protection. In theory it could also be based on the C drives bitlocker for a less secure, but still hardware level secured middle ground. Id have to think about it more.

      The other stuff mentioned is basically what it does locally in terms of OCR and recognition… just with proprietary local recipes.

      Thanks for your thoughts.

      • salarua@sopuli.xyzOP
        link
        fedilink
        English
        arrow-up
        3
        ·
        5 months ago

        IIUC it wouldn’t be able to be automatically started then, right? I mean I guess you could drag it to startup but it would need the password to start. From a security minded perspective that’s good, but from a user perspective kind of sucks.

        that’s true, but since this is a record of everything you’ve ever done, i feel this is the irreducible minimum for security. a separate password prompt would signal to the less technically-minded users that this is Serious

        Always forced to foreground makes it even less convenient and kind of odd.

        this is a design pattern i borrowed from Linux (my OS of choice). modern Linux apps require your explicit permission to run in the background, so most of them don’t even bother with running in the background at all. that said, i suppose it can run in the background, as long as the status indicator is sufficiently noticeable, but you’d have to go into the settings and flip that switch yourself

        I don’t see this functionality as being useful if you have to remember to turn it on.

        i imagine that it would become a habit, or you’d set it to run on startup. my use case would be turning it on for specific tasks like research or shopping, where you might only later remember that that one thing you saw was actually really valuable

        I figure the cryptfs could be a bitlocker volume with a different key than the base C drives key to get similar protection. In theory it could also be based on the C drives bitlocker for a less secure, but still hardware level secured middle ground.

        can a user-installed app do that?