Skip to content

[FIX] Image autoupload uploads images twice under certain conditions #4571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ ownCloud admins and users.

## Summary

* Bugfix - Changes in the automatic uploads algorithm to prevent duplications: [#3983](https://github.com/owncloud/android/issues/3983)
* Bugfix - Token request with Bearer returns error: [#4080](https://github.com/owncloud/android/issues/4080)
* Bugfix - Side menu collapses info in landscape: [#4513](https://github.com/owncloud/android/issues/4513)
* Bugfix - Content in Spaces not shown from third-party apps: [#4522](https://github.com/owncloud/android/issues/4522)
Expand All @@ -51,6 +52,16 @@ ownCloud admins and users.

## Details

* Bugfix - Changes in the automatic uploads algorithm to prevent duplications: [#3983](https://github.com/owncloud/android/issues/3983)

The timestamp for automatic uploads is now updated at the beginning of the
upload process instead of at the end. Additionally, the filter used in
AutomaticUploadsWorker to select the files to upload has been modified in order
to reduce time and charge when evaluating all files.

https://github.com/owncloud/android/issues/3983
https://github.com/owncloud/android/pull/4571

* Bugfix - Token request with Bearer returns error: [#4080](https://github.com/owncloud/android/issues/4080)

A new condition has been added into the network client to check if the network
Expand Down
7 changes: 7 additions & 0 deletions changelog/unreleased/4571
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Bugfix: Changes in the automatic uploads algorithm to prevent duplications

The timestamp for automatic uploads is now updated at the beginning of the upload process instead of at the end. Additionally, the filter
used in AutomaticUploadsWorker to select the files to upload has been modified in order to reduce time and charge when evaluating all files.

https://github.com/owncloud/android/issues/3983
https://github.com/owncloud/android/pull/4571
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ class ReleaseNotesViewModel(
subtitle = R.string.release_notes_4_6_0_subtitle_shares_space_docs_provider,
type = ReleaseNoteType.ENHANCEMENT
),
ReleaseNote(
title = R.string.release_notes_4_6_0_title_duplicated_automatic_uploads,
subtitle = R.string.release_notes_4_6_0_subtitle_duplicated_automatic_uploads,
type = ReleaseNoteType.BUGFIX
),
ReleaseNote(
title = R.string.release_notes_bugfixes_title,
subtitle = R.string.release_notes_bugfixes_subtitle,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ class SettingsPictureUploadsFragment : PreferenceFragmentCompat() {
picturesViewModel.enablePictureUploads(selectedAccount)
showAlertDialog(
title = getString(R.string.common_important),
message = getString(R.string.proper_pics_folder_warning_camera_upload)
message = getString(R.string.proper_pics_folder_warning_camera_upload, getString(R.string.app_name))
)
true
} else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ class SettingsVideoUploadsFragment : PreferenceFragmentCompat() {
videosViewModel.enableVideoUploads(selectedAccount)
showAlertDialog(
title = getString(R.string.common_important),
message = getString(R.string.proper_videos_folder_warning_camera_upload)
message = getString(R.string.proper_videos_folder_warning_camera_upload, getString(R.string.app_name))
)
true
} else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@
*
* @author Abel García de Prada
* @author Juan Carlos Garrote Gascón
* @author Jorge Aguado Recio
*
* Copyright (C) 2022 ownCloud GmbH.
* Copyright (C) 2025 ownCloud GmbH.
* <p>
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2,
Expand Down Expand Up @@ -122,9 +123,7 @@ class AutomaticUploadsWorker(
WorkManager.getInstance(appContext).cancelUniqueWork(AUTOMATIC_UPLOADS_WORKER)
}

private fun syncFolder(folderBackUpConfiguration: FolderBackUpConfiguration?) {
if (folderBackUpConfiguration == null) return

private fun syncFolder(folderBackUpConfiguration: FolderBackUpConfiguration) {
val syncType = when {
folderBackUpConfiguration.isPictureUploads -> SyncType.PICTURE_UPLOADS
folderBackUpConfiguration.isVideoUploads -> SyncType.VIDEO_UPLOADS
Expand All @@ -133,6 +132,7 @@ class AutomaticUploadsWorker(
}

val currentTimestamp = System.currentTimeMillis()
updateTimestamp(folderBackUpConfiguration, syncType, currentTimestamp)

val localPicturesDocumentFiles: List<DocumentFile> = getFilesReadyToUpload(
syncType = syncType,
Expand Down Expand Up @@ -166,7 +166,6 @@ class AutomaticUploadsWorker(
chargingOnly = folderBackUpConfiguration.chargingOnly
)
}
updateTimestamp(folderBackUpConfiguration, syncType, currentTimestamp)
}

private fun showNotification(
Expand Down Expand Up @@ -248,10 +247,14 @@ class AutomaticUploadsWorker(
val arrayOfLocalFiles = documentTree?.listFiles() ?: arrayOf()

val filteredList: List<DocumentFile> = arrayOfLocalFiles
.asSequence()
.filter {
it.lastModified() in lastSyncTimestamp..<currentTimestamp &&
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it would make more sense to move the MimetypeIconUtil check before the lastModified check. The rationale being that the mime type lookup is hopefully just a hash table lookup whereas the modification check has to touch the filesystem. This would save quite a bit of I/O for cases where not many files match (let's say automatic video upload and the folder containing photos & videos consists of only 1% of videos, in such a case you'd have 99% useless I/O calls -- I actually think that such a case is probably not uncommon, even if one has a 50:50 ratio of pictures vs photos you'd check half the file unnecessarily).

Copy link
Collaborator

@joragua joragua Jun 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right, but there are a lot of use cases... 🤔 As you said, you can have a folder in which the number of videos and photos is only a 1% and, with this filter, you will have 99% unnecessary calls. Also you can have a folder with a high number of photos and videos (99%) so, if we change the order there will be a lot of unnecessary calls anyway. In any case, @jesmrec is doing some tests so we will see what happens... But we will take into account your comments if something goes wrong. Thanks a lot for your opinion! 😄

MimetypeIconUtil.getBestMimeTypeByFilename(it.name).startsWith(syncType.prefixForType) &&
!it.name.orEmpty().startsWith(".")
}
.sortedBy { it.lastModified() }
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for sorting? Assuming lastModified is not cached on the file object this would again result in I/O (granted only a small amount since one usually doesn't have many files in 15 minutes). Since the updated timestamp is updated independent of the actual uploads, the order of files does not matter at all.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it works as you comment here but it could be... 🤔 The reason for sorting is: upload the files in the same order that were taken, so we can have an "history" in the uploads section...

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay, if one sees that as important then ordering makes sense.

.filter { it.lastModified() >= lastSyncTimestamp }
.filter { it.lastModified() < currentTimestamp }
.filter { MimetypeIconUtil.getBestMimeTypeByFilename(it.name).startsWith(syncType.prefixForType) }
.toList()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not knowing kotlin at all I am curious: What is the usecase for asSequence/toList here as opposed to the previous code that does without ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sequence is used to load and evaluate the files individually. The list loads all the files together and then the necessary checks are performed. The final .toList() is used to convert the sequence to a list (taking into account that the size of the list which contains the filtered files is smaller than the list of root folder)


Timber.i("Last sync ${syncType.name}: ${Date(lastSyncTimestamp)}")
Timber.i("CurrentTimestamp ${Date(currentTimestamp)}")
Expand Down
2 changes: 0 additions & 2 deletions owncloudApp/src/main/res/values-ar/strings.xml
Original file line number Diff line number Diff line change
Expand Up @@ -356,8 +356,6 @@
<string name="confirmation_disable_camera_uploads_title">تأكيد</string>
<string name="confirmation_disable_pictures_upload_message">هل أنت متأكد أنك ترغب في تعطيل هذه الميزة؟ لن يتم تحميل الصور المعلقة</string>
<string name="confirmation_disable_videos_upload_message">هل أنت متأكد أنك ترغب في تعطيل هذه الميزة؟ لن يتم تحميل مقاطع الفيديو المعلقة</string>
<string name="proper_pics_folder_warning_camera_upload">يُرجى التأكد من أن المجلد المحدد هو المكان الذي تحفظ فيه الكاميرا التي تستخدمها الصور الملتقطة. وإلا لن تتمكن الميزة من العثور على صورك. يُرجى مراعاة أن عملية تحميل الصور ستتم في غضون 15 دقيقة على الأقل من التقاطها.</string>
<string name="proper_videos_folder_warning_camera_upload">يُرجى التأكد من أن المجلد المحدد هو المكان الذي تحفظ فيه الكاميرا التي تستخدمها مقاطع الفيديو المسجلة. وإلا لن تتمكن الميزة من العثور على مقاطع الفيديو. يُرجى مراعاة أن عملية تحميل مقاطع الفيديو ستتم في غضون 15 دقيقة على الأقل من تسجيلها.</string>
<string name="conflict_description">حدث تعارض في ملف %1$s، انقر لحل هذه المشكلة</string>
<string name="conflict_message">ما الملفات التي ترغب في حفظها؟ إن حددت كلا الإصدارَين، فستتم إضافة رقم إلى اسم الملف المحلي.</string>
<string name="conflict_keep_both">احتفظ بالإصدارين</string>
Expand Down
2 changes: 0 additions & 2 deletions owncloudApp/src/main/res/values-bg-rBG/strings.xml
Original file line number Diff line number Diff line change
Expand Up @@ -356,8 +356,6 @@
<string name="placeholder_timestamp">2012/05/18 12:23 PM</string>
<string name="placeholder_media_time">12:23:45</string>
<string name="confirmation_disable_camera_uploads_title">Потвърждаване</string>
<string name="proper_pics_folder_warning_camera_upload">Моля уверете се, че избраната папка е мястото където камерата, която използвате запазва направените изображения. В противен случай функцията няма да може да ги открие. Имайте предвид, че изображенията ще бъдат качени най-малко 15 минути след заснемането им.</string>
<string name="proper_videos_folder_warning_camera_upload">Моля уверете се, че избраната папка е мястото където камерата, която използвате запазва записаните видеоклипове. В противен случай функцията няма да може да открие вашите видеоклипове. Имайте предвид, че видеоклиповете ще бъдат качени най-малко 15 минути след записването им.</string>
<string name="conflict_title">Файл в конфликт</string>
<string name="conflict_message">Кои файлове искате да запазите? Ако изберете и двете версии, локалния файл ще има числа добавени към името си.</string>
<string name="conflict_keep_both">Запазване и на двата</string>
Expand Down
2 changes: 0 additions & 2 deletions owncloudApp/src/main/res/values-ca/strings.xml
Original file line number Diff line number Diff line change
Expand Up @@ -340,8 +340,6 @@
<string name="confirmation_disable_camera_uploads_title">Confirma</string>
<string name="confirmation_disable_pictures_upload_message">Estàs segur que vols desactivar aquesta funció? Les fotos pendents no es carregaran.</string>
<string name="confirmation_disable_videos_upload_message">Estàs segur que vols desactivar aquesta funció? Les fotos pendents no es carregaran.</string>
<string name="proper_pics_folder_warning_camera_upload">Si us plau assegura\'t de que la carpeta que has seleccionat és on la càmara que estàs fent servir guarda les fotografies fetes. En cas contrari l\'aplicació no podrà detectar les vostres imatges. Tingueu en compte que les imatges es penjarant al menys 15 minuts després de fer-les.</string>
<string name="proper_videos_folder_warning_camera_upload">Si us plau assegura\'t de que la carpeta que has seleccionat és on la càmara que estàs fent servir guarda els videos fets. En cas contrari l\'aplicació no podrà detectar els vostres videos. Tingueu en compte que els videos es penjarant al menys 15 minuts després de fer-los.</string>
<string name="conflict_description">Hi ha un conflicte en l\'arxiu 1%1$s, clicka per arreglar-ho</string>
<string name="conflict_message">Quins fitxers vols mantenir? Si seleccioneu les dues versions, el fitxer local tindrà un número afegit al seu nom.</string>
<string name="conflict_keep_both">Mantén-los ambdós</string>
Expand Down
2 changes: 0 additions & 2 deletions owncloudApp/src/main/res/values-cs-rCZ/strings.xml
Original file line number Diff line number Diff line change
Expand Up @@ -410,8 +410,6 @@
<string name="confirmation_disable_camera_uploads_title">Potvrzení</string>
<string name="confirmation_disable_pictures_upload_message">Opravdu chcete tuto funkci deaktivovat? Nevyřízené obrázky nebudou nahrány.</string>
<string name="confirmation_disable_videos_upload_message">Opravdu chcete tuto funkci deaktivovat? Nevyřízená videa nebudou nahrána.</string>
<string name="proper_pics_folder_warning_camera_upload">Prosím ujistěte se, že vybraná složka obsahuje fotografie pořízené fotoaparátem. Jinak aplikace nebude schopna rozpoznat vaše fotografie. Mějte na paměti, že budou nahrány nejdříve 15 minut po jejich pořízení.</string>
<string name="proper_videos_folder_warning_camera_upload">Prosím ujistěte se, že vybraná složka obsahuje videa pořízená fotoaparátem. Jinak aplikace nebude schopna rozpoznat vaše videa. Mějte na paměti, že budou nahrána nejdříve 15 minut po jejich pořízení.</string>
<string name="conflict_title">Konflikt souboru</string>
<string name="conflict_description">Soubor %1$s obsahuje konflikt, kliknutím na tlačítko jej vyřešíte.</string>
<string name="conflict_message">Které soubory chcete ponechat? Pokud zvolíte obě verze, zkopírovaný soubor bude mít název doplněný o číslo.</string>
Expand Down
2 changes: 0 additions & 2 deletions owncloudApp/src/main/res/values-de-rCH/strings.xml
Original file line number Diff line number Diff line change
Expand Up @@ -350,8 +350,6 @@
<string name="confirmation_disable_camera_uploads_title">Bestätigen</string>
<string name="confirmation_disable_pictures_upload_message">Bist du dir sicher, dass du dieses Feature deaktivieren möchtest? Alle Bilder in der Warteschlange werden nicht hochgeladen</string>
<string name="confirmation_disable_videos_upload_message">Bist du dir sicher, dass du dieses Feature deaktivieren möchtest? Alle Videos in der Warteschlange werden nicht hochgeladen</string>
<string name="proper_pics_folder_warning_camera_upload">Bitte stelle sicher, dass sich der ausgewählte Ordner an der Stelle befindet, an der die von Ihnen verwendete Kamera die aufgenommenen Bilder speichert. Andernfalls kann die Funktion Ihre Bilder nicht erkennen. Beachte, dass die Bilder innerhalb von mindestens 15 Minuten nach der Aufnahme hochgeladen werden.</string>
<string name="proper_videos_folder_warning_camera_upload">Bitte stelle sicher, dass sich der ausgewählte Ordner an der Stelle befindet, an der die von Ihnen verwendete Kamera die aufgenommenen Videos speichert. Andernfalls kann die Funktion Ihre Videos nicht erkennen. Beachte, dass Videos innerhalb von mindestens 15 Minuten nach der Aufnahme hochgeladen werden.</string>
<string name="conflict_description">Ein Konflikt ist in der Datei %1$s aufgetreten, hier tippen um dies zu lösen</string>
<string name="conflict_message">Welche Datei möchtest du behalten? Wenn du beide Versionen auswählst, wird der lokalen Datei eine Zahl am Ende Ihres Dateiname angehangen.</string>
<string name="conflict_keep_both">Beide behalten</string>
Expand Down
2 changes: 0 additions & 2 deletions owncloudApp/src/main/res/values-de-rDE/strings.xml
Original file line number Diff line number Diff line change
Expand Up @@ -442,8 +442,6 @@
<string name="confirmation_disable_camera_uploads_title">Bestätigen</string>
<string name="confirmation_disable_pictures_upload_message">Sind Sie sicher, dass Sie dieses Feature deaktivieren möchten? Bilder in der Warteschlange werden nicht hochgeladen.</string>
<string name="confirmation_disable_videos_upload_message">Sind Sie sicher, dass Sie diese Funktion deaktivieren möchten? Videos in der Warteschlange werden nicht hochgeladen.</string>
<string name="proper_pics_folder_warning_camera_upload">Bitte stellen Sie sicher, dass der Ordner ausgewählt ist, in dem die Kamera die Bilder speichert. Anderenfalls werden Ihre Bilder nicht gefunden. Bitte beachten Sie, dass Bilder frühestens 15 Minuten, nachdem sie aufgenommen wurden, hochgeladen werden.</string>
<string name="proper_videos_folder_warning_camera_upload">Bitte stellen Sie sicher, dass der Ordner ausgewählt ist, in dem die Kamera die Videos speichert. Anderenfalls werden Ihre Videos nicht gefunden. Bitte beachten Sie, dass Videos frühestens 15 Minuten, nachdem sie aufgenommen wurden, hochgeladen werden.</string>
<string name="conflict_title">Dateikonflikt</string>
<string name="conflict_description">Ein Konflikt ist in der Datei %1$s aufgetreten. Hier tippen, um das Problem zu lösen.</string>
<string name="conflict_message">Welche Datei möchten Sie behalten? Wenn Sie beide Versionen auswählen, wird der lokalen Datei eine Zahl ans Ende ihres Dateinamens angehängt.</string>
Expand Down
2 changes: 0 additions & 2 deletions owncloudApp/src/main/res/values-de/strings.xml
Original file line number Diff line number Diff line change
Expand Up @@ -443,8 +443,6 @@
<string name="confirmation_disable_camera_uploads_title">Bestätigen</string>
<string name="confirmation_disable_pictures_upload_message">Soll dieses Feature wirklich deaktiviert werden? Die Bilder in der Warteschlange werden dann nicht hochgeladen.</string>
<string name="confirmation_disable_videos_upload_message">Soll dieses Feature wirklich deaktiviert werden? Die Videos in der Warteschlange werden dann nicht hochgeladen.</string>
<string name="proper_pics_folder_warning_camera_upload">Bitte stelle sicher, dass sich der ausgewählte Ordner an der Stelle befindet, an der die von Ihnen verwendete Kamera die aufgenommenen Bilder speichert. Andernfalls kann die Funktion Ihre Bilder nicht erkennen. Beachte, dass die Bilder innerhalb von mindestens 15 Minuten nach der Aufnahme hochgeladen werden.</string>
<string name="proper_videos_folder_warning_camera_upload">Bitte stelle sicher, dass sich der ausgewählte Ordner an der Stelle befindet, an der die von Ihnen verwendete Kamera die aufgenommenen Videos speichert. Andernfalls kann die Funktion Ihre Videos nicht erkennen. Beachte, dass Videos innerhalb von mindestens 15 Minuten nach der Aufnahme hochgeladen werden.</string>
<string name="conflict_title">Dateikonflikt</string>
<string name="conflict_description">Ein Konflikt ist in der Datei %1$s aufgetreten. Hier tippen, um das Problem zu lösen.</string>
<string name="conflict_message">Welche Datei möchtest du behalten? Wenn du beide Versionen auswählst, wird der lokalen Datei eine Zahl am Ende Ihres Dateiname angehangen.</string>
Expand Down
2 changes: 0 additions & 2 deletions owncloudApp/src/main/res/values-el/strings.xml
Original file line number Diff line number Diff line change
Expand Up @@ -348,8 +348,6 @@
<string name="confirmation_disable_camera_uploads_title">Επιβεβαίωση</string>
<string name="confirmation_disable_pictures_upload_message">Είστε βέβαιοι ότι θέλετε να απενεργοποιήσετε αυτήν τη λειτουργία; Οι φωτογραφίες που εκκρεμούν δεν θα μεταφορτωθούν</string>
<string name="confirmation_disable_videos_upload_message">Είστε βέβαιοι ότι θέλετε να απενεργοποιήσετε αυτήν τη λειτουργία; Τα βίντεο που εκκρεμούν δεν θα μεταφορτωθούν</string>
<string name="proper_pics_folder_warning_camera_upload">Βεβαιωθείτε ότι ο επιλεγμένος φάκελος είναι εκεί όπου η κάμερα που χρησιμοποιείτε αποθηκεύει τις φωτογραφίες που έχετε τραβήξει. Διαφορετικά, η λειτουργία δεν θα είναι σε θέση να ανιχνεύσει τις φωτογραφίες σας. Λάβετε υπόψη ότι οι φωτογραφίες θα μεταφορτωθούν τουλάχιστον 15 λεπτά μετά τη λήψη τους.</string>
<string name="proper_videos_folder_warning_camera_upload">Βεβαιωθείτε ότι ο επιλεγμένος φάκελος είναι εκεί όπου η κάμερα που χρησιμοποιείτε αποθηκεύει τα εγγεγραμμένα βίντεο. Διαφορετικά, η λειτουργία δεν θα είναι σε θέση να ανιχνεύσει τα βίντεό σας. Λάβετε υπόψη ότι τα βίντεο θα μεταφορτωθούν τουλάχιστον 15 λεπτά μετά την εγγραφή τους.</string>
<string name="conflict_description">Υπάρχει μια σύγκρουση στο αρχείο %1$s, πατήστε για να το λύσετε</string>
<string name="conflict_message">Ποια αρχεία θέλετε να διατηρήσετε; Αν επιλέξετε και τις δύο εκδόσεις, το τοπικό αρχείο θα έχει έναν αριθμό που έχει προστεθεί στο όνομά του.</string>
<string name="conflict_keep_both">Διατήρηση και των δύο</string>
Expand Down
Loading
Loading