gotosocial/internal/db/bundb/migrations
kim 311d9a1697 [chore] migration to update statuses.thread_id to be notnull (#4160)
# Description

This is quite a complex database migration that updates the `statuses.thread_id` column to be notnull, in order that statuses always be threaded, which will be useful in various pieces of upcoming work. This is unfortunately a migration that acts over the entire statuses table, and is quite complex in order to ensure that all existing statuses get correctly threaded together, and where possible fix any issues of statuses in the same thread having incorrect thread_ids.

TODO:
- ~~update testrig models to all be threaded~~
- ~~update code to ensure thread_id is always set~~
- ~~run on **a copy** of an sqlite production database~~
- ~~run on **a copy** of a postgres production database~~

## Checklist

- [x] I/we have read the [GoToSocial contribution guidelines](https://codeberg.org/superseriousbusiness/gotosocial/src/branch/main/CONTRIBUTING.md).
- [x] I/we have discussed the proposed changes already, either in an issue on the repository, or in the Matrix chat.
- [x] I/we have not leveraged AI to create the proposed changes.
- [x] I/we have performed a self-review of added code.
- [x] I/we have written code that is legible and maintainable by others.
- [x] I/we have commented the added code, particularly in hard-to-understand areas.
- [ ] I/we have made any necessary changes to documentation.
- [x] I/we have added tests that cover new code.
- [x] I/we have run tests and they pass locally with the changes.
- [x] I/we have run `go fmt ./...` and `golangci-lint run`.

Reviewed-on: https://codeberg.org/superseriousbusiness/gotosocial/pulls/4160
Co-authored-by: kim <grufwub@gmail.com>
Co-committed-by: kim <grufwub@gmail.com>
2025-05-26 15:33:42 +02:00
..
20211113114307_init [bugfix] Allow lowercase emoji shortcode in frontend (#1851) 2023-06-02 17:42:14 +02:00
20220214175650_media_cleanup [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20220315160814_admin_account_actions [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20220905150505_custom_emoji_updates [bugfix] Allow lowercase emoji shortcode in frontend (#1851) 2023-06-02 17:42:14 +02:00
20230328203024_migration_fix [feature] Account alias / move API + db models (#2518) 2024-01-16 16:22:44 +00:00
20230521105850_emoji_empty_domain_fix [chore] media and emoji refactoring (#3000) 2024-06-26 16:01:16 +01:00
20231002153327_add_status_polls [performance] convert enum strings to ints (#3558) 2024-11-25 14:48:59 +01:00
20231016113235_mute_status_thread [chore] migration to update statuses.thread_id to be notnull (#4160) 2025-05-26 15:33:42 +02:00
20240126064004_add_filters [bugfix] Fix filter title unique constraint (#3458) 2024-10-19 11:04:07 +02:00
20240318115336_account_settings [feature] New user sign-up via web page (#2796) 2024-04-11 11:45:53 +02:00
20240620074530_interaction_policy [chore] Tidy up previous interaction policy migrations (#4171) 2025-05-12 16:22:45 +00:00
20240715204203_media_pipeline_improvements [chore] media pipeline improvements (#3110) 2024-07-17 15:26:33 +00:00
20240809134448_interaction_requests_client_api [chore] Tidy up previous interaction policy migrations (#4171) 2025-05-12 16:22:45 +00:00
20240904084406_fedi_api_reject_interaction [performance] convert enum strings to ints (#3558) 2024-11-25 14:48:59 +01:00
20241022153016_domain_permission_draft_exclude [feature] Create/update/remove domain permission subscriptions (#3623) 2025-01-05 13:20:33 +01:00
20241022153016_domain_permission_subscriptions [bugfix] Store LastModified for domain perm subs + send as If-Modified-Since (#3655) 2025-01-20 10:56:00 +01:00
20241113152126_add_status_edits [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20241121121623_enum_strings_to_ints [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250106114512_replace_statuses_updatedat_with_editedat [chore] replace statuses.updated_at column with statuses.edited_at (#3636) 2025-01-08 11:29:23 +01:00
20250224105654_token_app_client_refactor [feature] Refactor tokens, allow multiple app redirect_uris (#3849) 2025-03-03 15:03:36 +00:00
20250226013442_add_status_content_type [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250305205820_content_warning_fixes [feature] Parse content warning to HTML, serialize via client API as plaintext (#3876) 2025-03-07 14:04:34 +00:00
20250321131230_relax_account_uri_uniqueness [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250324173534_2fa [feature] add TOTP two-factor authentication (2FA) (#3960) 2025-04-07 16:14:41 +02:00
20250415111056_thread_all_statuses [chore] migration to update statuses.thread_id to be notnull (#4160) 2025-05-26 15:33:42 +02:00
20211113114307_init.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220214175650_media_cleanup.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220305130328_database_optimizations.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220315160814_admin_account_actions.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220502113806_add_missing_indexes.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220506110822_add_account_raw_note.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20220511165212_add_account_raw_note_fix.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20220518123621_additional_indexing.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220612091800_duplicated_media_cleanup.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220710153020_fix_slow_web_profile_queries.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220804120132_account_default_post_format.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20220823140228_user_custom_css.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20220827085121_assign_missing_in_reply_to_uris.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220830014732_regenerate_indexes_for_follows.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220903141016_store_gifs_as_image.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20220905150505_custom_emoji_updates.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220916122701_emojis_in_accounts.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20220922142408_shared_inbox_delivery.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20220926115233_indexes.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20221006114842_add_rss_functionality.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20221011125732_refetch_updated_emojis.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20221031145649_emoji_categories.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20221103203553_add_external_id.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20221108142419_create_account_tombstones.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20221220134514_mp4_jiggery_pokery.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20230105171144_report_model.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230126170719_format_to_content_type.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20230202212700_rename_account_webfingered_to_fetched.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
20230221150957_status_pin_client_api.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230328203024_migration_fix.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230405130021_status_fave_unique_constraints.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230430105735_fieldwork.go [feature] Enable federation in/out of profile PropertyValue fields (#1722) 2023-05-09 11:16:10 +01:00
20230511181430_add_status_fetched_at.go [feature] status refetch support (#1690) 2023-05-12 11:15:54 +02:00
20230515173919_lists.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230521105850_emoji_empty_domain_fix.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230620103932_search_updates.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230711214815_account_notes.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230713025939_markers_api.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230718161520_hashtaggery.go [feature] Hashtag federation (in/out), hashtag client API endpoints (#2032) 2023-07-31 15:47:35 +02:00
20230724100000_emoji_cleanup.go [feature/performance] support uncaching remote emoji + scheduled cleanup functions (#1987) 2023-07-24 13:14:13 +01:00
20230815164500_rules_model.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230817174700_add_report_rule_ids.go [feature] Instance rules (#2125) 2023-08-19 14:33:15 +02:00
20230821075342_attachment_cleanup_updates.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230828101322_admin_action_locking.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20230905133904_remote_pubkey_expiry.go [feature] Allow admins to expire remote public keys; refetch expired keys on demand (#2183) 2023-09-12 10:43:12 +01:00
20230908083121_allowlist.go.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20231002153327_add_status_polls.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20231016113235_mute_status_thread.go [chore] migration to update statuses.thread_id to be notnull (#4160) 2025-05-26 15:33:42 +02:00
20231029143819_remove_suspended_at_index.go Remove account_suspended_at_idx to resolve slow query issues (#2310) 2023-10-31 13:43:33 +00:00
20231110142330_small_poll_table_tweaks.go [bugfix] Update poll delete/update db queries (#2361) 2023-11-14 12:43:27 +00:00
20231128140847_remove_duplicate_indices.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20231130103643_fix_index_whoopsie.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20231208152242_sqlite_analyze.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20231212144715_add_header_filters.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20231215115920_add_status_poll_index.go [bugfix] use a much shorter refresh limit for statuses with polls (#2453) 2023-12-15 15:24:39 +01:00
20231227110845_instance_description_updates.go [feature] Parse instance descriptors as markdown, show T&C on /about (#2481) 2024-01-05 13:39:31 +01:00
20240103170945_moved_to_also_known_as.go [feature] Account alias / move API + db models (#2518) 2024-01-16 16:22:44 +00:00
20240114112637_postgres_header_filter_fix.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240126064004_add_filters.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240129170725_moved_to_also_known_as.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240220204526_add_statuses_mentions_is_null_or_empty_idx.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240228113608_readd_statuses_account_id_id_idx.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240310120046_fix_empty_move_id.go [bugfix] Fix whitespace move_id issue (#2742) 2024-03-10 13:27:31 +01:00
20240318115336_account_settings.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240320114447_preset_css_themes.go [feature] User-selectable preset CSS themes for accounts (#2777) 2024-03-25 17:32:24 +00:00
20240401130338_sign_up.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240402113305_nil_domain_fix.go [bugfix] Set domain for empty-domain Friendica accounts (#2800) 2024-04-02 13:40:54 +01:00
20240414122348_account_stats_model.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240426122821_pageable_admin_accounts.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240528071620_add_user_mutes.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240611190733_add_conversations.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240613091853_drop_unused_media_columns.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240617134210_add_worker_tasks_table.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240620074530_interaction_policy.go [chore] Tidy up previous interaction policy migrations (#4171) 2025-05-12 16:22:45 +00:00
20240712005536_add_advanced_migrations.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240715204203_media_pipeline_improvements.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240716151327_interaction_policy.go [feature] Interaction requests client api + settings panel (#3215) 2024-08-24 11:49:37 +02:00
20240722222556_remove_boost_content.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240725211933_add_followed_tags.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240809134448_interaction_requests_client_api.go [chore] Tidy up previous interaction policy migrations (#4171) 2025-05-12 16:22:45 +00:00
20240903212842_add_exclusive_lists.go [feature] Implement exclusive lists (#3280) 2024-09-09 15:56:58 -07:00
20240904084406_fedi_api_reject_interaction.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240906144432_unauthed_visibility.go.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20240924222938_add_instance_custom_css.go [feature] Add global instance CSS customization setting (#3352) 2024-12-02 12:24:48 +01:00
20241005100438_interaction_requests_pending_tweak.go [bugfix] Update select of pending interaction requests to account for potential nil URI (#3392) 2024-10-05 12:27:53 +02:00
20241011115713_pending_approval_fix.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20241018151036_filter_unique_fix.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20241022153016_domain_permission_draft_exclude.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20241113151042_remove_mention_updated_at.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20241113152126_add_status_edits.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20241121121623_enum_strings_to_ints.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20241124012635_add_vapid_key_pairs.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20241124012636_add_web_push_subscriptions.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20241203124608_remove_media_attachment_updated_at.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250106114512_replace_statuses_updatedat_with_editedat.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250106165529_cleanup_dropped_edits.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250119112745_domain_permission_subscriptions.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250119112746_domain_perm_sub_caching_tweaks.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250124164400_domain_perm_sub_migration_fix.go [chore] Rewrite all remaining Github links 2025-04-27 13:40:22 +02:00
20250126162825_top_level_mention_replies_fix.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250131184755_add_web_push_subscription_policy.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250224105654_token_app_client_refactor.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250226013442_add_status_content_type.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250305205820_content_warning_fixes.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250310094108_statuses_count_query_optimize.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250310144102_application_management.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250314120945_add_gallery_web_layout.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250318093828_statuses_public_timeline_reindex.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250321131230_relax_account_uri_uniqueness.go [bugfix] Fix migration unsetting bot flag (#4098) 2025-04-30 11:17:20 +00:00
20250324173534_2fa.go [feature] Move to code.superseriousbusiness.org 2025-04-26 15:38:43 +02:00
20250415111056_thread_all_statuses.go [chore] migration to update statuses.thread_id to be notnull (#4160) 2025-05-26 15:33:42 +02:00
main.go [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
README.md [chore/docs] Replace specific year range of copyright notice (#2520) 2024-01-13 16:33:53 +01:00
util.go [chore] migration to update statuses.thread_id to be notnull (#4160) 2025-05-26 15:33:42 +02:00

Migrations

How do I write a migration file?

See here

As a template, take one of the existing migration files and modify it, or use the below code snippet:

// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program.  If not, see <http://www.gnu.org/licenses/>.

package migrations

import (
    "context"

    "github.com/uptrace/bun"
)

func init() {
    up := func(ctx context.Context, db *bun.DB) error {
        return db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error {
            // your logic here
            return nil
        })
    }

    down := func(ctx context.Context, db *bun.DB) error {
        return db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error {
            // your logic here
            return nil
        })
    }

    if err := Migrations.Register(up, down); err != nil {
        panic(err)
    }
}

File format

Bun requires a very specific format: 14 digits, then letters or underscores.

You can use the following bash command on your branch to generate a suitable migration filename.

echo "$(date --utc +%Y%m%d%H%M%S | head -c 14)_$(git rev-parse --abbrev-ref HEAD).go"

Rules of thumb

  1. DON'T DROP TABLES!!!!!!!!
  2. Don't make something NOT NULL if it's likely to already contain null fields.