Commit Graph
22 Commits
Author SHA1 Message Date
c4fb82858e enhance(lfs): report status and body on non-JSON LFS error responses (#40331)
A push mirror or migration pointed at a remote that has LFS disabled
fails with a confusing error like `readObjectStart: expect { or n, but
found N ... |Not Found`. That comes from decoding the batch endpoint's
plain-text "Not Found" response as JSON.

Read the response body before decoding and, when it is not valid LFS
JSON, return the HTTP status together with the raw body so the message
points at the real cause. Responses that are proper LFS JSON errors
behave as before.

Fixes: https://github.com/go-gitea/gitea/issues/34897

Assisted-by: Claude Code:claude-opus-4-8

---------

Co-authored-by: amalsaidov <amalsaidov@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-11-12 07:10:04 +00:00
silverwind 8abda62312 test: consolidate Go tests (#39962)
Consolidate all Go tests. Coverage is identical to before. Go tests are
around 5% faster while being 7% smaller. No non-test code is touched.
2026-10-24 13:48:19 +00:00
Shudhanshu Singhandwxiaoguang b5e53210c2 fix(lfs): send the default branch as ref in batch download requests (#39891)
Fixes https://github.com/go-gitea/gitea/issues/38085

---------

Signed-off-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2026-10-21 03:12:41 -07:00
Lunny Xiao 61b1a39efe chore: Move import path from code.gitea.io/gitea to gitea.dev (#37873) 2026-05-26 15:49:31 -07:00
wxiaoguang 0f668145e9 Drop json-iterator dependency (#35544) 2025-09-28 22:30:28 +08:00
151ef80e28 use experimental go json v2 library (#35392)
details: https://pkg.go.dev/encoding/json/v2

---------

Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-09-28 08:03:36 +00:00
TheFox0x7andwxiaoguang ee3c82f874 Enable addtional linters (#34085)
enable mirror, usestdlibbars and perfsprint 
part of: https://github.com/go-gitea/gitea/issues/34083

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-01 10:14:01 +00:00
cc1fdc84ca Use test context in tests and new loop system in benchmarks (#33648)
Replace all contexts in tests with go1.24 t.Context()

---------

Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-20 09:57:40 +00:00
wxiaoguang 41b4ef825d Use 8 as default value for git lfs concurrency (#32421) 2024-11-05 13:10:57 +00:00
Royce Remerandwxiaoguang 54146e62c0 Make LFS http_client parallel within a batch. (#32369)
Signed-off-by: Royce Remer <royceremer@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-04 04:49:08 +00:00
Royce Remer df805d6ed0 Support legacy _links LFS batch responses (#31513)
Support legacy _links LFS batch response.

Fixes #31512.

This is backwards-compatible change to the LFS client so that, upon
mirroring from an upstream which has a batch api, it can download
objects whether the responses contain the `_links` field or its
successor the `actions` field. When Gitea must fallback to the legacy
`_links` field a logline is emitted at INFO level which looks like this:
```
...s/lfs/http_client.go:188:performOperation() [I] <LFSPointer ee95d0a27ccdfc7c12516d4f80dcf144a5eaf10d0461d282a7206390635cdbee:160> is using a deprecated batch schema response!
```

I've only run `test-backend` with this code, but added a new test to
cover this case. Additionally I have a fork with this change deployed
which I've confirmed syncs LFS from Gitea<-Artifactory (which has legacy
`_links`) as well as from Gitea<-Gitea (which has the modern `actions`).

Signed-off-by: Royce Remer <royceremer@gmail.com>
2024-06-28 08:42:57 +00:00
Zoupers Zou e25d6960b5 Fix #31185 try fix lfs download from bitbucket failed (#31201)
Fix #31185
2024-06-11 22:22:28 +00:00
Chongyi Zhengandwxiaoguang 9631958a82 Refactor lfs requests (#26783)
- Refactor lfs request code
- The original code uses `performRequest` function to create the
request, uses a callback to modify the request, and then send the
request.
- Now it's replaced with `createRequest` that only creates request and
`performRequest` that only sends the request.
- Reuse `createRequest` and `performRequest` in `http_client.go` and
`transferadapter.go`

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-18 08:40:50 +00:00
f1173d6879 Use more specific test methods (#24265)
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-04-22 17:56:27 -04:00
e81ccc406b Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
KN4CK3R 3f280f89e7 Update HTTP status codes to modern codes (#18063)
* 2xx/3xx/4xx/5xx -> http.Status...
* http.StatusFound -> http.StatusTemporaryRedirect
* http.StatusMovedPermanently -> http.StatusPermanentRedirect
2022-03-23 12:54:07 +08:00
6543 54e9ee37a7 format with gofumpt (#18184)
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
2022-01-20 18:46:10 +01:00
Eng Zer Junandtechknowlogick f2e7d5477f refactor: move from io/ioutil to io and os package (#17109)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-22 13:38:34 +08:00
Lunny Xiao 9f31f3aa8a Add an abstract json layout to make it's easier to change json library (#16528)
* Add an abstract json layout to make it's easier to change json library

* Fix import

* Fix import sequence

* Fix blank lines

* Fix blank lines
2021-07-24 18:03:58 +02:00
KN4CK3R 440039c0cc Add push to remote mirror repository (#15157)
* Added push mirror model.

* Integrated push mirror into queue.

* Moved methods into own file.

* Added basic implementation.

* Mirror wiki too.

* Removed duplicated method.

* Get url for different remotes.

* Added migration.

* Unified remote url access.

* Add/Remove push mirror remotes.

* Prevent hangs with missing credentials.

* Moved code between files.

* Changed sanitizer interface.

* Added push mirror backend methods.

* Only update the mirror remote.

* Limit refs on push.

* Added UI part.

* Added missing table.

* Delete mirror if repository gets removed.

* Changed signature. Handle object errors.

* Added upload method.

* Added "upload" unit tests.

* Added transfer adapter unit tests.

* Send correct headers.

* Added pushing of LFS objects.

* Added more logging.

* Simpler body handling.

* Process files in batches to reduce HTTP calls.

* Added created timestamp.

* Fixed invalid column name.

* Changed name to prevent xorm auto setting.

* Remove table header im empty.

* Strip exit code from error message.

* Added docs page about mirroring.

* Fixed date.

* Fixed merge errors.

* Moved test to integrations.

* Added push mirror test.

* Added test.
2021-06-14 19:20:43 +02:00
KN4CK3R 3607f79d78 Fixed assert statements. (#16089) 2021-06-07 07:27:09 +02:00
KN4CK3Randzeripath c03e488e14 Add LFS Migration and Mirror (#14726)
* Implemented LFS client.

* Implemented scanning for pointer files.

* Implemented downloading of lfs files.

* Moved model-dependent code into services.

* Removed models dependency. Added TryReadPointerFromBuffer.

* Migrated code from service to module.

* Centralised storage creation.

* Removed dependency from models.

* Moved ContentStore into modules.

* Share structs between server and client.

* Moved method to services.

* Implemented lfs download on clone.

* Implemented LFS sync on clone and mirror update.

* Added form fields.

* Updated templates.

* Fixed condition.

* Use alternate endpoint.

* Added missing methods.

* Fixed typo and make linter happy.

* Detached pointer parser from gogit dependency.

* Fixed TestGetLFSRange test.

* Added context to support cancellation.

* Use ReadFull to probably read more data.

* Removed duplicated code from models.

* Moved scan implementation into pointer_scanner_nogogit.

* Changed method name.

* Added comments.

* Added more/specific log/error messages.

* Embedded lfs.Pointer into models.LFSMetaObject.

* Moved code from models to module.

* Moved code from models to module.

* Moved code from models to module.

* Reduced pointer usage.

* Embedded type.

* Use promoted fields.

* Fixed unexpected eof.

* Added unit tests.

* Implemented migration of local file paths.

* Show an error on invalid LFS endpoints.

* Hide settings if not used.

* Added LFS info to mirror struct.

* Fixed comment.

* Check LFS endpoint.

* Manage LFS settings from mirror page.

* Fixed selector.

* Adjusted selector.

* Added more tests.

* Added local filesystem migration test.

* Fixed typo.

* Reset settings.

* Added special windows path handling.

* Added unit test for HTTPClient.

* Added unit test for BasicTransferAdapter.

* Moved into util package.

* Test if LFS endpoint is allowed.

* Added support for git://

* Just use a static placeholder as the displayed url may be invalid.

* Reverted to original code.

* Added "Advanced Settings".

* Updated wording.

* Added discovery info link.

* Implemented suggestion.

* Fixed missing format parameter.

* Added Pointer.IsValid().

* Always remove model on error.

* Added suggestions.

* Use channel instead of array.

* Update routers/repo/migrate.go

* fmt

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-04-08 18:25:57 -04:00