update moved files around like crazy

[trapd00r]

So I went ahead and started a beet update yesterday, thinking that my
custom rules would be in effect, i.e label:ocremix.

label:ocremix:               +game/+ocremix/$base_name

I imported those files a while back like this:

beet import -sAW --set label=ocremix .

And the tracks were moved to exactly where I wanted them.

Now, when I did the update, however, beets created over a thousand new
artist directories, placing the individual tracks inside them. Just like
the ocremix label rule didn't exist anymore.

What I had to do now was a

beet modify ':ocremix' label=ocremix

and the files are back where they belong. However, the next time I'll
run the update this will happen again? It's not just those files either,
I have a mess to clean up now.

I know that I can use the -M flag for update, but there are things I
want to move around according to new rules in my config.

So the question is: why didn't beet respect the custom rules that I
have setup, or why was the label tag no longer present when I was
running the update?

Here's my full configuration:

# vim:synmaxcol=500:expandtab:fdm=marker:fdl=0:
#      ${HOME}/etc/beets/config.yaml
#   ‗‗‗‗‗‗‗‗‗‗‗‗ ‗‗‗‗‗‗ ‗‗‗‗‗‗‗‗ ‗‗‗‗‗‗‗‗‗‗‗
#         owner  Magnus Woldrich <[email protected]>
#         btime  2021-05-13 10:31:39
#         mtime  2021-05-31 16:38:07
#   permissions  You are free to use things you may find useful here.
#                Please improve and share.
#           git  http://github.com/trapd00r/configs/  (up-to-date)
#           url  http://japh.se
#           irc  [email protected] #vim #perl #beets
#   ‗‗‗‗‗‗‗‗‗‗‗‗ ‗‗‗‗‗‗‗‗‗‗‗‗‗ ‗‗‗‗ ‗‗‗‗ ‗‗‗‗

#< what does it look like
## Alanis Morissette │2020│ Reckoning [Single, WEB, MP3]
## Alanis Morissette │2020│ Such Pretty Forks in the Road [WEB, FLAC]
## Anna Ternheim │2003│ My Secret [EP, CD, MP3]
## Anna Ternheim │2004│ Somebody Outside [CD, MP3]
## Anna Ternheim │2005│ Shoreline EP [EP, CD, MP3]
##  ├── 01 Shoreline (radio Version).mp3
##  ├── 02 Little Lies.mp3
##  ├── 03 China Girl.mp3
##  ├── 04 When Tomorrow Comes.mp3
##  ├── 05 Anywhere I Lay My Head.mp3
##  └── cover.jpg

## A/Anna Ternheim/Anna Ternheim │2003│ My Secret [EP, CD, MP3]
#   '- 01 My Secret.mp3
#   '- 02 All For Me.mp3
#   '- 03 A Voice To Calm You Down.mp3
#   '- 04 I Say No (gotland Version).mp3
#   '- 05 Wedding Song (demo Version).mp3
#   '- cover.jpg
#>

#< core options
directory:  /mnt/music8/+TAGGED
library:    ~/var/beets/beets202105.db
pluginpath: /usr/lib/python3.9/site-packages/beetsplug/

# files matching these patterns are deleted from source after import
clutter: ["Thumbs.DB", ".DS_Store", ".m3u", ".pls",
          ".jpg", ".nfo", ".txt", ".log", ".gif",
         ]

replace:
    '[\\]': ''
    '[_]': '-'
    '[/]': '-'
    '^\.': ''
    '[\x00-\x1f]': ''
    '[<>:"\?\*\|]': ''
    '\.$': ''
    '\s+$': ''
    '^\s+': ''
    '^-': ''

ignore: [".*", "*~", "System Volume Information"]

art_filename:          cover # cover.jpg
#asciify_paths:         yes
format_item:           $path
per_disc_numbering:    true
sort_album:            path+
sort_case_insensitive: yes
sort_item:             path+
threaded:              yes
timeout:               5.0
verbose:               no

# <importer
import:
  write:            yes
  default_action:   apply
#  remove causes a crash: https://github.com/beetbox/beets/issues/716
  duplicate_action: keep
  non_rec_action:   ask
  autotag:          yes
  move:             yes
#  quiet_fallback:   asis # when using the -q flag
  quiet_fallback:   skip # when using the -q flag

  # Either yes or no, controlling whether imported directories are recorded
  # and whether these recorded directories are skipped. This corresponds to
  # the -i flag to beet import.
  incremental:      yes
#>
#< plugins
#plugins: [ # mosaic
#  'fetchart',   'discogs',  'fromfilename', 'inline',     'smartplaylist',
#  'ftintitle',  'info',     'lastgenre',    'lastimport', 'thumbnails',
#  'mpdupdate',  'mpdstats', 'rewrite',      'duplicates', 'missing',
#  'extrafiles', 'edit',     'lyrics',       'mpdqueue',
#]

plugins: [
  'bucket',
  'discogs',
  'duplicates',
#  'edit',
  'extrafiles',
  'fetchart',
  'fromfilename',
  'ftintitle',
  'info',
  'inline',
  'lastgenre',
  'lastimport',
#  'lyrics', # takes forever
  'missing',
#  'mpdqueue',
#  'mpdstats',
#  'mpdupdate',
  'rewrite',
#  'smartplaylist', # only run it once in a while
]


bucket:
  bucket_alpha:
  - '#-!'
  - '0-9'
  - 'A'
  - 'B'
  - 'C'
  - 'D'
  - 'E'
  - 'F'
  - 'G'
  - 'H'
  - 'I'
  - 'J'
  - 'K'
  - 'L'
  - 'M'
  - 'N'
  - 'O'
  - 'P'
  - 'Q'
  - 'R'
  - 'S'
  - 'T'
  - 'U'
  - 'V'
  - 'W'
  - 'X'
  - 'Y'
  - 'Z'
  bucket_alpha_regex:
    '#-!': ^[^0-9a-zA-ZåÅäÄöÖ]
  bucket_year: []
  extrapolate: no

bpd:
  host: 127.0.0.1
  port: 6601

edit:
  itemfields: track title artist album year
  albumfields: track title artist albumartist album year

extrafiles:
  patterns:
    single_tracks:
      - '+tracks/'
      - '_tracks/'
    single_live:
      - '_live'
      - '+live'
  paths:
    single_tracks: $artist/+tracks
    single_live: $artist/+live


rewrite:
  album .*Sommar i P1:               P1 Sommar
  album .*Sommar och Vinter i P1.*:  P1 Sommar
  album Söndagsintervjun:            P1 Söndagsintervjun
  album .*Musikguiden i p3:          P3 Musikguiden
  album Jukeboxen i p4:              P4 Jukeboxen
  album Musikspecial i p4:           P4 Musikspecial
  albumartist Various Artists:       VA
  artist 江海迦:                     Aga
  artist Whoo Kid:                   DJ Whoo Kid
  artist Looptroop.*:                Looptroop
  artist T.R:                        Öris
  artist Organismen:                 Organism 12
  artist Gms.*:                      GMS
  artist (tupac|2[pP]ac).*:          2pac
  artist .*weird Al.*:               Weird Al Yankovic
  artist .*Green Lantern.*:          DJ Green Lantern
  artist .ingenting.:                Ingenting
  artist Sin[eé]ad O.Connor.*:       Sinéad O'Connor
  artist .*Suzanne.*Vega:            Suzanne Vega
  artist .*1[23]00 mic.*:            1200 Micrograms
  artist elin (ruth)? sigvardsson:   Elin Sigvardsson
  artist elin ruth:                  Elin Sigvardsson
  artist ^Game$:                     The Game
  artist ^Ken$:                      Ken Ring
  artist Special D:                  Special D.
  artist Danne W.*:                  Sjätte Sinnet
  artist Sjatte Sinnet:              Sjätte Sinnet
  artist Ante Barazza:               Sjätte Sinnet

mpd:
  host: localhost
  port: 6600
#  music_directory: /mnt/music8

thumbnails:
  auto: yes # default

lastfm:
  user: betbot

lastgenre:
  auto: yes # default
  canonical: yes
  force: no
  source: artist

ftintitle:
  auto: yes # default

smartplaylist:
# it's better to set save_absolute_paths_in_playlist option in mpd.conf
#  relative_to: ~/mp3/
  playlist_dir: ~/mp3/_playlists
  playlists:
    - name: '+all.m3u'
      query: ''

    - name: 'eminem.m3u'
      query: 'artist:Eminem'

#    - name: 'sm %lower{$genre}.m3u'
#      query: ''

    - name: '$year.m3u'
      query: 'year::(199[0-9]|200[0-9]|201[0-9])'

    - name: '+decent.m3u'
      query: 'play_count:1..'

    - name: '+wow.m3u'
      query: 'play_count:5..'

    - name: 'psychedelic'
      query: 'genre:psychedelic'

musicbrainz:
  searchlimit: 10

missing:
  format: "$path"

lyrics:
  auto: yes
#  fallback: ''

fetchart:
  auto: yes
  sources: coverart itunes amazon albumart wikipedia google

embedart:
  auto: yes
#>
#< path setup
aunique:
  disambuguators: media mastering label catalognum albumdisambig releasegroupdisambig

match:
  preferred:
    media: ['CD', 'Digital Media|File', 'Vinyl']

paths:
# https://www.japh.se/2021/05/23/custom-beet-path-rules-for-record-labels.html
#    mb_trackid::^$:              +unmatched/
    label:8bitpeoples:           8/8bitpeoples/%if{$hasyear,${year}} %title{$albumartist} - %title{$album}/${padded_tracknr} %title{$title}
    label:whoa.nu:               W/Whoa.nu/$base_name
    label:randombastards:        R/Randombastards/$base_name
    label:Norrköping:            S/Sjätte Sinnet/+tracks/$base_name
    label:Rosamannen:            +live/+rosamannen/$base_name
    label:frizon.info:           F/Frizon.info/$base_name
    label:"Masters of Hardcore": M/Masters of Hardcore/$moh_catalog %if{$hasyear,${year}} %title{$album}/${padded_tracknr} %title{$artist} - %title{$title}
    label:gamesoundtrack:        +game/+ost/%if{$hasyear,│${year}│}$album%aunique{} [$alb_type$media_type$format]/${padded_tracknr} $artist - %title{$title}
    label:ocremix:               +game/+ocremix/$base_name
    label:overlooked:            +game/+overlookedremix/$base_name
    label:amigaremix:            +game/+amigaremix/$base_name
    label:gamealbums:            +game/$albumartist - %title{$album}%if{$hasyear, (${year})}/${padded_tracknr} $artist - %title{$title}
    label:game:                  +game/+tracks/$base_name
    label:radio:                 +radio/%title{$artist}/%title{$album}/%title{$title}
    label:mixtape:               +mixtape/%title{$album}%if{$hasyear, (${year})}/${padded_tracknr} $artist - %title{$title}
    label:demo:                  %bucket{$albumartist, alpha}/%title{$first_artist}/+demo/$base_name
    label:live:                  %bucket{$albumartist, alpha}/%title{$first_artist}/+live/%title{$albumartist}%if{$hasyear, │${year}│} %title{$album} [$alb_type$media_type$format]/${padded_tracknr} %title{$title}
#    default:                     %upper{%left{$albumartist,1}}/%title{$first_artist}/%title{$albumartist}%if{$hasyear, │${year}│} %title{$album} [$alb_type$media_type$format]/${padded_tracknr} %title{$title}
    default:                     %bucket{$albumartist, alpha}/%title{$first_artist}/%title{$albumartist}%if{$hasyear, │${year}│} %title{$album} [$alb_type$media_type$format]/${padded_tracknr} %title{$title}
    albumtype:soundtrack:        +OST/%if{$hasyear,│${year}│}$album%aunique{} [$alb_type$media_type$format]/${padded_tracknr} $artist - %title{$title}
    comp:                        +VA/%title{$album}%if{$hasyear, (${year})}/${padded_tracknr} $artist - %title{$title}

# https://www.japh.se/2021/05/26/how-to-add-singletons-to-artist-dir-correctly-in-beets.html
    singleton:                   %upper{%left{$artist,1}}/$artist/+tracks/$base_name

# I want bootlegs etc in the same setup as default
    albumtype:other:             %upper{%left{$albumartist,1}}/%title{$first_artist}/%title{$albumartist}%if{$hasyear, │${year}│} %title{$album} [$alb_type$media_type$format]/${padded_tracknr} %title{$title}
#>
#< inline fun!
item_fields:
  # pad track number with zero if < 10
  padded_tracknr: "'{:02n}'.format(track)"
#  moh_catalog: catalognum.replace(" ", "")

# capture first artist as primary artist to avoid directories like this:
# · B/Britney Spears/
# · B/Britney Spears feat Madonna/
# · B/Britney Spears vs Metallica/
# > https://github.com/beetbox/beets/issues/3176
# > https://www.japh.se/2021/06/01/capture-primary-artist-as-a-separate-field-in-beets.html
#
# handles:
# · Artist,
# · Artist &
# · Artist feat
# · Artist feat.
# · Artist featuring
# · Artist ft.
# · Artist vs
# · Artist vs.
# · Artist &
#
# The idea is to use $first_artist in the beginning of the path format
# like so:
#
# %title{$first_artist}/%title{$albumartist}
#
# which will put 'Jennifer Lopez feat. Pitbull' inside the main Jennifer
# Lopez directory, but still keep the feat. part in the directory name
# inside it.
#
# J/Jennifer Lopez Feat. Pitbull/Jennifer Lopez Feat. Pitbull │2012│ Dance Again [Single, WEB, MP3]/01 Dance Again.mp3
# -> J/Jennifer Lopez/Jennifer Lopez Feat. Pitbull │2012│ Dance Again [Single, WEB, MP3]/01 Dance Again.mp3

  first_artist: |
    import re
    return re.split(',|\s+(feat(.?|uring)|&|(Vs|Ft).)', albumartist, 1, flags=re.IGNORECASE)[0]

# file basename for singletons import - import as is, minus the extension.
  base_name: |
    import os.path
    base = os.path.basename(path)
    return os.path.splitext(base)[0]

album_fields:
    alb_status: |
        # MB returns 4 values describing how "offical" a release is, they are:
        # Official, Promotional, Bootleg, and Pseudo-Release
        # We only note the middle two.
        # https://musicbrainz.org/doc/Release#Status
        if 'Promo' in albumstatus:
            return 'Promo'
        elif 'Bootleg' in albumstatus:
            return 'Bootleg'
        else:
            return None
    # Check if https://github.com/beetbox/beets/issues/2200 affects below
    alb_type: |
        alb_types = ""
        albumtypes_list = {
            'ep': 'EP, ',
            'single': 'Single, ',
            'live': 'Live, ',
            'remix': 'Remix, ',
            'dj-mix': 'DJ-mix, ',
            'mixtape/street': 'Mixtape, ',
            'interview': 'Interview, ',
        }
        for key, value in albumtypes_list.items():
            if albumtype == key:
                alb_types += str(value)
        if alb_types is not None:
            return alb_types
        else:
            return None

    media_type: |
        # https://musicbrainz.org/doc/Release/Format
        # Lets Merge the variations of the same medium into the main medium name (Opinonated)
        media_list = {
            '12" Vinyl': 'VINYL, ',
            '10" Vinyl': 'VINYL, ',
            '7" Vinyl': 'VINYL, ',
            'Cassette': 'CASSETTE, ',
            'Digital Media': 'WEB, ',
            'CD': 'CD, ',
            'File': 'WEB, ',
        }
        # Lets omit these instead of converging them under a similar label like above (Opinonated)
        media_types_to_omit = ['Enhanced CD', 'CDDA', 'Blu-spec CD', 'SHM-CD', 'HQCD', '']
        if items[0].media in media_list:
            return str(media_list[items[0].media])
        elif items[0].media in media_types_to_omit:
            return None
        else:
            return str(items[0].media)

    hasyear: 1 if year > 0 else 0
#>
#< autotagger
# To control how tolerant the autotagger is of differences, use the
# strong_rec_thresh option, which reflects the distance threshold below
# which beets will make a “strong recommendation” that the metadata
# be used.
#
# default is 0.04
match:
  strong_rec_thresh: 0.10
  medium_rec_thresh: 0.25
  ignored:           missing_tracks unmatched_tracks
  ignored_media:     ['Data CD', 'DVD', 'DVD-Video', 'Blu-ray', 'HD-DVD',
                      'VCD', 'SVCD', 'UMD', 'VHS']
#>
#< ui
ui:
  color: yes
  colors:
    text_success:         green
    text_warning:         yellow
    text_error:           red
    text_highlight:       blue
    text_highlight_minor: lightgray
    action_default:       turquoise
    action:               blue
#>

[trapd00r]

This is weird. Why doesn't beet write when I import even though I have:

import:
  write: yes

I use the -A since I don't want beet to try musicbrainz for these files.

 beet import -sA --set artist="Sveriges Radio" --set label=radio *(.)

A beet info, no label here:

/mnt/music8/+TAGGED/+radio/Sveriges Radio/Vm 1994 I Usa (stronger Sound).mp3
       art: False
    artist: Lasse Granqvist & Ralf Edström
  bitdepth: 0
   bitrate: 119605
  channels: 2
  comments: www.dvdvideosoft.com
    format: MP3
    genres: 
    length: 985.9192743764172
samplerate: 44100
     title: VM 1994 i USA (Stronger Sound)
     track: 1

But when I issue a write command:

% beet write -p "Vm 1994 I" 

/mnt/music8/+TAGGED/+radio/Sveriges Radio/Vm 1994 I Usa (stronger Sound).mp3
  artist: Lasse Granqvist & Ralf Edström -> Sveriges Radio
  genre:  -> Comedy
  label:  -> radio

[trapd00r]

It doesn't write even if I explicitly pass the -w flag to import. But a beet write afterwards work.