Changelog for v1.14.5 (#16450)

Once #16449 is merged I think we should release 1.14.5. There are a couple of
security fixes and the broken #16268 is annoying enough that we should just release
things.

Signed-off-by: Andrew Thornton <art27@cantab.net>

.drone.yml

@@ -522,7 +522,7 @@ steps:
     image: plugins/s3:1
     settings:
       acl: public-read
-      bucket: releases
+      bucket: gitea-artifacts
       endpoint: https://storage.gitea.io
       path_style: true
       source: "dist/release/*"
@@ -543,7 +543,7 @@ steps:
     image: plugins/s3:1
     settings:
       acl: public-read
-      bucket: releases
+      bucket: gitea-artifacts
       endpoint: https://storage.gitea.io
       path_style: true
       source: "dist/release/*"
@@ -618,7 +618,7 @@ steps:
     image: plugins/s3:1
     settings:
       acl: public-read
-      bucket: releases
+      bucket: gitea-artifacts
       endpoint: https://storage.gitea.io
       path_style: true
       source: "dist/release/*"
@@ -709,7 +709,7 @@ steps:
 
   - name: publish
     pull: always
-    image: plugins/docker:linux-amd64
+    image: techknowlogick/drone-docker:latest
     settings:
       auto_tag: true
       auto_tag_suffix: linux-amd64
@@ -726,7 +726,7 @@ steps:
         - pull_request
 
   - name: publish-rootless
-    image: plugins/docker:linux-amd64
+    image: techknowlogick/drone-docker:latest
     settings:
       dockerfile: Dockerfile.rootless
       auto_tag: true
@@ -764,7 +764,7 @@ trigger:
 steps:
   - name: dryrun
     pull: always
-    image: plugins/docker:linux-arm64
+    image: techknowlogick/drone-docker:latest
     settings:
       dry_run: true
       repo: gitea/gitea
@@ -806,7 +806,7 @@ steps:
 
   - name: publish
     pull: always
-    image: plugins/docker:linux-arm64
+    image: techknowlogick/drone-docker:latest
     settings:
       auto_tag: true
       auto_tag_suffix: linux-arm64
@@ -826,7 +826,7 @@ steps:
         - pull_request
 
   - name: publish-rootless
-    image: plugins/docker:linux-arm64
+    image: techknowlogick/drone-docker:latest
     settings:
       dockerfile: Dockerfile.rootless
       auto_tag: true

CHANGELOG.md

@@ -4,6 +4,81 @@ This changelog goes through all the changes that have been made in each release
 without substantial changes to our git log; to see the highlights of what has
 been added to each release, please refer to the [blog](https://blog.gitea.io).
 
+## [1.14.5](https://github.com/go-gitea/gitea/releases/tag/v1.14.5) - 2021-07-16
+
+* SECURITY
+  * Hide mirror passwords on repo settings page (#16022) (#16355)
+  * Update bluemonday to v1.0.15 (#16379) (#16380)
+* BUGFIXES
+  * Retry rename on lock induced failures (#16435) (#16439)
+  * Validate issue index before querying DB (#16406) (#16410)
+  * Fix crash following ldap authentication update (#16447) (#16449)
+* ENHANCEMENTS
+  * Redirect on bad CSRF instead of presenting bad page (#14937) (#16378)
+
+## [1.14.4](https://github.com/go-gitea/gitea/releases/tag/v1.14.4) - 2021-07-06
+
+* BUGFIXES
+  * Fix relative links in postprocessed images (#16334) (#16340)
+  * Fix list_options GetStartEnd (#16303) (#16305)
+  * Fix API to use author for commits instead of committer (#16276) (#16277)
+  * Handle misencoding of login_source cfg in mssql (#16268) (#16275)
+  * Fixed issues not updated by commits (#16254) (#16261)
+  * Improve efficiency in FindRenderizableReferenceNumeric and getReference (#16251) (#16255)
+  * Use html.Parse rather than html.ParseFragment (#16223) (#16225)
+  * Fix milestone counters on new issue (#16183) (#16224)
+  * reqOrgMembership calls need to be preceded by reqToken (#16198) (#16219)
+
+## [1.14.3](https://github.com/go-gitea/gitea/releases/tag/v1.14.3) - 2021-06-10
+
+* SECURITY
+  * Encrypt migration credentials at rest (#15895) (#16187)
+  * Only check access tokens if they are likely to be tokens (#16164) (#16171)
+  * Add missing SameSite settings for the i_like_gitea cookie (#16037) (#16039)
+  * Fix setting of SameSite on cookies (#15989) (#15991)
+* API
+  * Repository object only count releases as releases (#16184) (#16190)
+  * EditOrg respect RepoAdminChangeTeamAccess option (#16184) (#16190)
+  * Fix overly strict edit pr permissions (#15900) (#16081)
+* BUGFIXES
+  * Run processors on whole of text (#16155) (#16185)
+  * Class `issue-keyword` is being incorrectly stripped off spans (#16163) (#16172)
+  * Fix language switch for install page (#16043) (#16128)
+  * Fix bug on getIssueIDsByRepoID (#16119) (#16124)
+  * Set self-adjusting deadline for connection writing (#16068) (#16123)
+  * Fix http path bug (#16117) (#16120)
+  * Fix data URI scramble (#16098) (#16118)
+  * Merge all deleteBranch as one function and also fix bug when delete branch don't close related PRs (#16067) (#16097)
+  * git migration: don't prompt interactively for clone credentials (#15902) (#16082)
+  * Fix case change in ownernames (#16045) (#16050)
+  * Don't manipulate input params in email notification (#16011) (#16033)
+  * Remove branch URL before IssueRefURL (#15968) (#15970)
+  * Fix layout of milestone view (#15927) (#15940)
+  * GitHub Migration, migrate draft releases too (#15884) (#15888)
+  * Close the gitrepo when deleting the repository (#15876) (#15887)
+  * Upgrade xorm to v1.1.0 (#15869) (#15885)
+  * Fix blame row height alignment (#15863) (#15883)
+  * Fix error message when saving generated LOCAL_ROOT_URL config (#15880) (#15882)
+  * Backport Fix LFS commit finder not working (#15856) (#15874)
+  * Stop calling WriteHeader in Write (#15862) (#15873)
+  * Add timeout to writing to responses (#15831) (#15872)
+  * Return go-get info on subdirs (#15642) (#15871)
+  * Restore PAM user autocreation functionality (#15825) (#15867)
+  * Fix truncate utf8 string (#15828) (#15854)
+  * Fix bound address/port for caddy's certmagic library (#15758) (#15848)
+  * Upgrade unrolled/render to v1.1.1 (#15845) (#15846)
+  * Queue manager FlushAll can loop rapidly - add delay (#15733) (#15840)
+  * Tagger can be empty, as can Commit and Author - tolerate this (#15835) (#15839)
+  * Set autocomplete off on branches selector (#15809) (#15833)
+  * Add missing error to Doctor log (#15813) (#15824)
+  * Move restore repo to internal router and invoke from command to avoid open the same db file or queues files (#15790) (#15816)
+* ENHANCEMENTS
+  * Removable media support to snap package (#16136) (#16138)
+  * Move sans-serif fallback font higher than emoji fonts (#15855) (#15892)
+* DOCKER
+  * Only write config in environment-to-ini if there are changes (#15861) (#15868)
+  * Only offer hostcertificates if they exist (#15849) (#15853)
+
 ## [1.14.2](https://github.com/go-gitea/gitea/releases/tag/v1.14.2) - 2021-05-08
 
 * API

cmd/embedded.go

@@ -19,6 +19,7 @@ import (
 	"code.gitea.io/gitea/modules/public"
 	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/modules/templates"
+	"code.gitea.io/gitea/modules/util"
 
 	"github.com/gobwas/glob"
 	"github.com/urfave/cli"
@@ -271,7 +272,7 @@ func extractAsset(d string, a asset, overwrite, rename bool) error {
 	} else if !fi.Mode().IsRegular() {
 		return fmt.Errorf("%s already exists, but it's not a regular file", dest)
 	} else if rename {
-		if err := os.Rename(dest, dest+".bak"); err != nil {
+		if err := util.Rename(dest, dest+".bak"); err != nil {
 			return fmt.Errorf("Error creating backup for %s: %v", dest, err)
 		}
 		// Attempt to respect file permissions mask (even if user:group will be set anew)

cmd/restore_repo.go

@@ -5,15 +5,12 @@
 package cmd
 
 import (
-	"context"
-	"strings"
+	"errors"
+	"net/http"
 
 	"code.gitea.io/gitea/modules/log"
-	"code.gitea.io/gitea/modules/migrations"
-	"code.gitea.io/gitea/modules/migrations/base"
+	"code.gitea.io/gitea/modules/private"
 	"code.gitea.io/gitea/modules/setting"
-	"code.gitea.io/gitea/modules/storage"
-	pull_service "code.gitea.io/gitea/services/pull"
 
 	"github.com/urfave/cli"
 )
@@ -50,70 +47,18 @@ wiki, issues, labels, releases, release_assets, milestones, pull_requests, comme
 }
 
 func runRestoreRepository(ctx *cli.Context) error {
-	if err := initDB(); err != nil {
-		return err
-	}
+	setting.NewContext()
 
-	log.Trace("AppPath: %s", setting.AppPath)
-	log.Trace("AppWorkPath: %s", setting.AppWorkPath)
-	log.Trace("Custom path: %s", setting.CustomPath)
-	log.Trace("Log path: %s", setting.LogRootPath)
-	setting.InitDBConfig()
-
-	if err := storage.Init(); err != nil {
-		return err
-	}
-
-	if err := pull_service.Init(); err != nil {
-		return err
-	}
-
-	var opts = base.MigrateOptions{
-		RepoName: ctx.String("repo_name"),
-	}
-
-	if len(ctx.String("units")) == 0 {
-		opts.Wiki = true
-		opts.Issues = true
-		opts.Milestones = true
-		opts.Labels = true
-		opts.Releases = true
-		opts.Comments = true
-		opts.PullRequests = true
-		opts.ReleaseAssets = true
-	} else {
-		units := strings.Split(ctx.String("units"), ",")
-		for _, unit := range units {
-			switch strings.ToLower(unit) {
-			case "wiki":
-				opts.Wiki = true
-			case "issues":
-				opts.Issues = true
-			case "milestones":
-				opts.Milestones = true
-			case "labels":
-				opts.Labels = true
-			case "releases":
-				opts.Releases = true
-			case "release_assets":
-				opts.ReleaseAssets = true
-			case "comments":
-				opts.Comments = true
-			case "pull_requests":
-				opts.PullRequests = true
-			}
-		}
-	}
-
-	if err := migrations.RestoreRepository(
-		context.Background(),
+	statusCode, errStr := private.RestoreRepo(
 		ctx.String("repo_dir"),
 		ctx.String("owner_name"),
 		ctx.String("repo_name"),
-	); err != nil {
-		log.Fatal("Failed to restore repository: %v", err)
-		return err
+		ctx.StringSlice("units"),
+	)
+	if statusCode == http.StatusOK {
+		return nil
 	}
 
-	return nil
+	log.Fatal("Failed to restore repository: %v", errStr)
+	return errors.New(errStr)
 }

cmd/web.go

@@ -175,7 +175,7 @@ func setPort(port string) error {
 
 		cfg.Section("server").Key("LOCAL_ROOT_URL").SetValue(defaultLocalURL)
 		if err := cfg.SaveTo(setting.CustomConf); err != nil {
-			return fmt.Errorf("Error saving generated JWT Secret to custom config: %v", err)
+			return fmt.Errorf("Error saving generated LOCAL_ROOT_URL to custom config: %v", err)
 		}
 	}
 	return nil

cmd/web_letsencrypt.go

@@ -6,6 +6,7 @@ package cmd
 
 import (
 	"net/http"
+	"strconv"
 	"strings"
 
 	"code.gitea.io/gitea/modules/log"
@@ -22,6 +23,15 @@ func runLetsEncrypt(listenAddr, domain, directory, email string, m http.Handler)
 	// TODO: these are placeholders until we add options for each in settings with appropriate warning
 	enableHTTPChallenge := true
 	enableTLSALPNChallenge := true
+	altHTTPPort := 0
+	altTLSALPNPort := 0
+
+	if p, err := strconv.Atoi(setting.PortToRedirect); err == nil {
+		altHTTPPort = p
+	}
+	if p, err := strconv.Atoi(setting.HTTPPort); err == nil {
+		altTLSALPNPort = p
+	}
 
 	magic := certmagic.NewDefault()
 	magic.Storage = &certmagic.FileStorage{Path: directory}
@@ -30,6 +40,9 @@ func runLetsEncrypt(listenAddr, domain, directory, email string, m http.Handler)
 		Agreed:                  setting.LetsEncryptTOS,
 		DisableHTTPChallenge:    !enableHTTPChallenge,
 		DisableTLSALPNChallenge: !enableTLSALPNChallenge,
+		ListenHost:              setting.HTTPAddr,
+		AltTLSALPNPort:          altTLSALPNPort,
+		AltHTTPPort:             altHTTPPort,
 	})
 
 	magic.Issuer = myACME

contrib/environment-to-ini/environment-to-ini.go

@@ -110,6 +110,8 @@ func runEnvironmentToIni(c *cli.Context) error {
 	}
 	cfg.NameMapper = ini.SnackCase
 
+	changed := false
+
 	prefix := c.String("prefix") + "__"
 
 	for _, kv := range os.Environ() {
@@ -143,15 +145,21 @@ func runEnvironmentToIni(c *cli.Context) error {
 				continue
 			}
 		}
+		oldValue := key.Value()
+		if !changed && oldValue != value {
+			changed = true
+		}
 		key.SetValue(value)
 	}
 	destination := c.String("out")
 	if len(destination) == 0 {
 		destination = setting.CustomConf
 	}
-	err = cfg.SaveTo(destination)
-	if err != nil {
-		return err
+	if destination != setting.CustomConf || changed {
+		err = cfg.SaveTo(destination)
+		if err != nil {
+			return err
+		}
 	}
 	if c.Bool("clear") {
 		for _, kv := range os.Environ() {

custom/conf/app.example.ini

@@ -281,6 +281,10 @@ HTTP_PORT = 3000
 ; PORT_TO_REDIRECT.
 REDIRECT_OTHER_PORT = false
 PORT_TO_REDIRECT = 80
+; Timeout for any write to the connection. (Set to 0 to disable all timeouts.)
+PER_WRITE_TIMEOUT = 30s
+; Timeout per Kb written to connections.
+PER_WRITE_PER_KB_TIMEOUT = 30s
 ; Permission for unix socket
 UNIX_SOCKET_PERMISSION = 666
 ; Local (DMZ) URL for Gitea workers (such as SSH update) accessing web service.

docker/root/etc/s6/openssh/setup

@@ -24,9 +24,29 @@ if [ ! -f /data/ssh/ssh_host_ecdsa_key ]; then
     ssh-keygen -t ecdsa -b 256 -f /data/ssh/ssh_host_ecdsa_key -N "" > /dev/null
 fi
 
+if [ -e /data/ssh/ssh_host_ed25519_cert ]; then
+  SSH_ED25519_CERT=${SSH_ED25519_CERT:-"/data/ssh/ssh_host_ed25519_cert"}
+fi
+
+if [ -e /data/ssh/ssh_host_rsa_cert ]; then
+  SSH_RSA_CERT=${SSH_RSA_CERT:-"/data/ssh/ssh_host_rsa_cert"}
+fi
+
+if [ -e /data/ssh/ssh_host_ecdsa_cert ]; then
+  SSH_ECDSA_CERT=${SSH_ECDSA_CERT:-"/data/ssh/ssh_host_ecdsa_cert"}
+fi
+
+if [ -e /data/ssh/ssh_host_dsa_cert ]; then
+  SSH_DSA_CERT=${SSH_DSA_CERT:-"/data/ssh/ssh_host_dsa_cert"}
+fi
+
 if [ -d /etc/ssh ]; then
     SSH_PORT=${SSH_PORT:-"22"} \
     SSH_LISTEN_PORT=${SSH_LISTEN_PORT:-"${SSH_PORT}"} \
+    SSH_ED25519_CERT="${SSH_ED25519_CERT:+"HostCertificate "}${SSH_ED25519_CERT}" \
+    SSH_RSA_CERT="${SSH_RSA_CERT:+"HostCertificate "}${SSH_RSA_CERT}" \
+    SSH_ECDSA_CERT="${SSH_ECDSA_CERT:+"HostCertificate "}${SSH_ECDSA_CERT}" \
+    SSH_DSA_CERT="${SSH_DSA_CERT:+"HostCertificate "}${SSH_DSA_CERT}" \
     envsubst < /etc/templates/sshd_config > /etc/ssh/sshd_config
 
     chmod 0644 /etc/ssh/sshd_config

docker/root/etc/templates/sshd_config

@@ -8,13 +8,13 @@ ListenAddress ::
 LogLevel INFO
 
 HostKey /data/ssh/ssh_host_ed25519_key
-HostCertificate /data/ssh/ssh_host_ed25519_cert
+${SSH_ED25519_CERT}
 HostKey /data/ssh/ssh_host_rsa_key
-HostCertificate /data/ssh/ssh_host_rsa_cert
+${SSH_RSA_CERT}
 HostKey /data/ssh/ssh_host_ecdsa_key
-HostCertificate /data/ssh/ssh_host_ecdsa_cert
+${SSH_ECDSA_CERT}
 HostKey /data/ssh/ssh_host_dsa_key
-HostCertificate /data/ssh/ssh_host_dsa_cert
+${SSH_DSA_CERT}
 
 AuthorizedKeysFile .ssh/authorized_keys
 AuthorizedPrincipalsFile .ssh/authorized_principals

docs/Makefile

@@ -31,4 +31,4 @@ update: $(THEME)
 $(THEME): $(THEME)/theme.toml
 $(THEME)/theme.toml:
 	mkdir -p $$(dirname $@)
-	curl -s $(ARCHIVE) | tar xz -C $$(dirname $@)
+	curl -L -s $(ARCHIVE) | tar xz -C $$(dirname $@)

docs/content/doc/advanced/config-cheat-sheet.en-us.md

@@ -237,6 +237,9 @@ Values containing `#` or `;` must be quoted using `` ` `` or `"""`.
    most cases you do not need to change the default value. Alter it only if
    your SSH server node is not the same as HTTP node. Do not set this variable
    if `PROTOCOL` is set to `unix`.
+- `PER_WRITE_TIMEOUT`: **30s**: Timeout for any write to the connection. (Set to 0 to
+   disable all timeouts.)
+- `PER_WRITE_PER_KB_TIMEOUT`: **10s**: Timeout per Kb written to connections.
 
 - `DISABLE_SSH`: **false**: Disable SSH feature when it's not available.
 - `START_SSH_SERVER`: **false**: When enabled, use the built-in SSH server.
@@ -260,6 +263,9 @@ Values containing `#` or `;` must be quoted using `` ` `` or `"""`.
 - `SSH_KEY_TEST_PATH`: **/tmp**: Directory to create temporary files in when testing public keys using ssh-keygen, default is the system temporary directory.
 - `SSH_KEYGEN_PATH`: **ssh-keygen**: Path to ssh-keygen, default is 'ssh-keygen' which means the shell is responsible for finding out which one to call.
 - `SSH_EXPOSE_ANONYMOUS`: **false**: Enable exposure of SSH clone URL to anonymous visitors, default is false.
+- `SSH_PER_WRITE_TIMEOUT`: **30s**: Timeout for any write to the SSH connections. (Set to
+  0 to disable all timeouts.)
+- `SSH_PER_WRITE_PER_KB_TIMEOUT`: **10s**: Timeout per Kb written to SSH connections.
 - `MINIMUM_KEY_SIZE_CHECK`: **true**: Indicate whether to check minimum key size with corresponding type.
 
 - `OFFLINE_MODE`: **false**: Disables use of CDN for static files and Gravatar for profile pictures.

go.mod

@@ -86,7 +86,7 @@ require (
 	github.com/mgechev/revive v1.0.3
 	github.com/mholt/acmez v0.1.3 // indirect
 	github.com/mholt/archiver/v3 v3.5.0
-	github.com/microcosm-cc/bluemonday v1.0.7
+	github.com/microcosm-cc/bluemonday v1.0.15
 	github.com/miekg/dns v1.1.40 // indirect
 	github.com/minio/md5-simd v1.1.2 // indirect
 	github.com/minio/minio-go/v7 v7.0.10
@@ -122,7 +122,7 @@ require (
 	github.com/unknwon/com v1.0.1
 	github.com/unknwon/i18n v0.0.0-20200823051745-09abd91c7f2c
 	github.com/unknwon/paginater v0.0.0-20200328080006-042474bd0eae
-	github.com/unrolled/render v1.1.0
+	github.com/unrolled/render v1.1.1
 	github.com/urfave/cli v1.22.5
 	github.com/willf/bitset v1.1.11 // indirect
 	github.com/xanzy/go-gitlab v0.44.0
@@ -136,10 +136,10 @@ require (
 	go.uber.org/multierr v1.6.0 // indirect
 	go.uber.org/zap v1.16.0 // indirect
 	golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83
-	golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4
+	golang.org/x/net v0.0.0-20210614182718-04defd469f4e
 	golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93
-	golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44
-	golang.org/x/text v0.3.5
+	golang.org/x/sys v0.0.0-20210423082822-04245dca01da
+	golang.org/x/text v0.3.6
 	golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba // indirect
 	golang.org/x/tools v0.1.0
 	gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0-20150716171945-2caba252f4dc // indirect
@@ -149,7 +149,7 @@ require (
 	mvdan.cc/xurls/v2 v2.2.0
 	strk.kbt.io/projects/go/libravatar v0.0.0-20191008002943-06d1c002b251
 	xorm.io/builder v0.3.9
-	xorm.io/xorm v1.0.7
+	xorm.io/xorm v1.1.0
 )
 
 replace github.com/hashicorp/go-version => github.com/6543/go-version v1.2.4

go.sum

@@ -127,8 +127,9 @@ github.com/asaskevich/govalidator v0.0.0-20180720115003-f9ffefc3facf/go.mod h1:l
 github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
 github.com/asaskevich/govalidator v0.0.0-20200108200545-475eaeb16496/go.mod h1:oGkLhpf+kjZl6xBf758TQhh5XrAeiJv/7FRz/2spLIg=
 github.com/asaskevich/govalidator v0.0.0-20200428143746-21a406dcc535/go.mod h1:oGkLhpf+kjZl6xBf758TQhh5XrAeiJv/7FRz/2spLIg=
-github.com/asaskevich/govalidator v0.0.0-20200907205600-7a23bdc65eef h1:46PFijGLmAjMPwCCCo7Jf0W6f9slllCkkv7vyc1yOSg=
 github.com/asaskevich/govalidator v0.0.0-20200907205600-7a23bdc65eef/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
+github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d h1:Byv0BzEl3/e6D5CLfI0j/7hiIEtvGVFPCZ7Ei2oq8iQ=
+github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
 github.com/aws/aws-lambda-go v1.13.3/go.mod h1:4UKl9IzQMoD+QF79YdCuzCwp8VbmG4VAQwij/eHl5CU=
 github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
 github.com/aws/aws-sdk-go v1.34.28/go.mod h1:H7NKnBqNVzoTJpGfLrQkkD+ytBA93eiDYi/+8rV9s48=
@@ -830,8 +831,8 @@ github.com/mholt/acmez v0.1.3 h1:J7MmNIk4Qf9b8mAGqAh4XkNeowv3f1zW816yf4zt7Qk=
 github.com/mholt/acmez v0.1.3/go.mod h1:8qnn8QA/Ewx8E3ZSsmscqsIjhhpxuy9vqdgbX2ceceM=
 github.com/mholt/archiver/v3 v3.5.0 h1:nE8gZIrw66cu4osS/U7UW7YDuGMHssxKutU8IfWxwWE=
 github.com/mholt/archiver/v3 v3.5.0/go.mod h1:qqTTPUK/HZPFgFQ/TJ3BzvTpF/dPtFVJXdQbCmeMxwc=
-github.com/microcosm-cc/bluemonday v1.0.7 h1:6yAQfk4XT+PI/dk1ZeBp1gr3Q2Hd1DR0O3aEyPUJVTE=
-github.com/microcosm-cc/bluemonday v1.0.7/go.mod h1:HOT/6NaBlR0f9XlxD3zolN6Z3N8Lp4pvhp+jLS5ihnI=
+github.com/microcosm-cc/bluemonday v1.0.15 h1:J4uN+qPng9rvkBZBoBb8YGR+ijuklIMpSOZZLjYpbeY=
+github.com/microcosm-cc/bluemonday v1.0.15/go.mod h1:ZLvAzeakRwrGnzQEvstVzVt3ZpqOF2+sdFr0Om+ce30=
 github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
 github.com/miekg/dns v1.1.30/go.mod h1:KNUDUusw/aVsxyTYZM1oqvCicbwhgbNgztCETuNZ7xM=
 github.com/miekg/dns v1.1.40 h1:pyyPFfGMnciYUk/mXpKkVmeMQjfXqt3FAJ2hy7tPiLA=
@@ -996,6 +997,8 @@ github.com/quasoft/websspi v1.0.0 h1:5nDgdM5xSur9s+B5w2xQ5kxf5nUGqgFgU4W0aDLZ8Mw
 github.com/quasoft/websspi v1.0.0/go.mod h1:HmVdl939dQ0WIXZhyik+ARdI03M6bQzaSEKcgpFmewk=
 github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
 github.com/rcrowley/go-metrics v0.0.0-20190826022208-cac0b30c2563/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
+github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 h1:OdAsTTz6OkFY5QxjkYwrChwuRruF69c169dPK26NUlk=
+github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
 github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
 github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
 github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
@@ -1113,10 +1116,8 @@ github.com/unknwon/i18n v0.0.0-20200823051745-09abd91c7f2c h1:679/gJXwrsHC3RATr0
 github.com/unknwon/i18n v0.0.0-20200823051745-09abd91c7f2c/go.mod h1:+5rDk6sDGpl3azws3O+f+GpFSyN9GVr0K8cvQLQM2ZQ=
 github.com/unknwon/paginater v0.0.0-20200328080006-042474bd0eae h1:ihaXiJkaca54IaCSnEXtE/uSZOmPxKZhDfVLrzZLFDs=
 github.com/unknwon/paginater v0.0.0-20200328080006-042474bd0eae/go.mod h1:1fdkY6xxl6ExVs2QFv7R0F5IRZHKA8RahhB9fMC9RvM=
-github.com/unrolled/render v1.0.3 h1:baO+NG1bZSF2WR4zwh+0bMWauWky7DVrTOfvE2w+aFo=
-github.com/unrolled/render v1.0.3/go.mod h1:gN9T0NhL4Bfbwu8ann7Ry/TGHYfosul+J0obPf6NBdM=
-github.com/unrolled/render v1.1.0 h1:gvpR9hHxTt6DcGqRYuVVFcfd8rtK+nyEPUJN06KB57Q=
-github.com/unrolled/render v1.1.0/go.mod h1:gN9T0NhL4Bfbwu8ann7Ry/TGHYfosul+J0obPf6NBdM=
+github.com/unrolled/render v1.1.1 h1:FpzNzkvlJQIlVdVaqeVBGWiCS8gpbmjtrKpDmCn6p64=
+github.com/unrolled/render v1.1.1/go.mod h1:gN9T0NhL4Bfbwu8ann7Ry/TGHYfosul+J0obPf6NBdM=
 github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
 github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
 github.com/urfave/cli v1.22.5 h1:lNq9sAHXK2qfdI8W+GRItjCEkI+2oR4d+MEHy1CKXoU=
@@ -1321,9 +1322,8 @@ golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwY
 golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
-golang.org/x/net v0.0.0-20210331212208-0fccb6fa2b5c/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
-golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4 h1:4nGaVu0QrbjT/AK2PRLuQfQuh6DJve+pELhqTdAj3x0=
-golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
+golang.org/x/net v0.0.0-20210614182718-04defd469f4e h1:XpT3nA5TvE525Ne3hInMh6+GETgn27Zfm9dxsThnX2Q=
+golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20181106182150-f42d05182288/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -1419,8 +1419,8 @@ golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44 h1:Bli41pIlzTzf3KEY06n+xnzK/BESIg2ze4Pgfh/aI8c=
-golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da h1:b3NXsE2LusjYGGjL5bxEVZZORm/YEFFrWFjR8eFrw/c=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
@@ -1430,8 +1430,9 @@ golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3
 golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.5 h1:i6eZZ+zk0SOf0xgBpEpPD18qWcJda6q1sxt3S0kzyUQ=
 golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
+golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -1502,6 +1503,7 @@ golang.org/x/tools v0.0.0-20200928182047-19e03678916f/go.mod h1:z6u4i615ZeAfBE4X
 golang.org/x/tools v0.0.0-20200929161345-d7fc70abf50f/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
 golang.org/x/tools v0.0.0-20201022035929-9cf592e881e9/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/tools v0.0.0-20201124115921-2c860bdd6e78/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20201125231158-b5590deeca9b/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
@@ -1668,6 +1670,33 @@ honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt
 honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
 honnef.co/go/tools v0.0.1-2020.1.4 h1:UoveltGrhghAA7ePc+e+QYDHXrBps2PqFZiHkGR/xK8=
 honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
+modernc.org/cc/v3 v3.31.5-0.20210308123301-7a3e9dab9009 h1:u0oCo5b9wyLr++HF3AN9JicGhkUxJhMz51+8TIZH9N0=
+modernc.org/cc/v3 v3.31.5-0.20210308123301-7a3e9dab9009/go.mod h1:0R6jl1aZlIl2avnYfbfHBS1QB6/f+16mihBObaBC878=
+modernc.org/ccgo/v3 v3.9.0 h1:JbcEIqjw4Agf+0g3Tc85YvfYqkkFOv6xBwS4zkfqSoA=
+modernc.org/ccgo/v3 v3.9.0/go.mod h1:nQbgkn8mwzPdp4mm6BT6+p85ugQ7FrGgIcYaE7nSrpY=
+modernc.org/httpfs v1.0.6 h1:AAgIpFZRXuYnkjftxTAZwMIiwEqAfk8aVB2/oA6nAeM=
+modernc.org/httpfs v1.0.6/go.mod h1:7dosgurJGp0sPaRanU53W4xZYKh14wfzX420oZADeHM=
+modernc.org/libc v1.7.13-0.20210308123627-12f642a52bb8/go.mod h1:U1eq8YWr/Kc1RWCMFUWEdkTg8OTcfLw2kY8EDwl039w=
+modernc.org/libc v1.8.0 h1:Pp4uv9g0csgBMpGPABKtkieF6O5MGhfGo6ZiOdlYfR8=
+modernc.org/libc v1.8.0/go.mod h1:U1eq8YWr/Kc1RWCMFUWEdkTg8OTcfLw2kY8EDwl039w=
+modernc.org/mathutil v1.1.1/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
+modernc.org/mathutil v1.2.2 h1:+yFk8hBprV+4c0U9GjFtL+dV3N8hOJ8JCituQcMShFY=
+modernc.org/mathutil v1.2.2/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
+modernc.org/memory v1.0.4 h1:utMBrFcpnQDdNsmM6asmyH/FM9TqLPS7XF7otpJmrwM=
+modernc.org/memory v1.0.4/go.mod h1:nV2OApxradM3/OVbs2/0OsP6nPfakXpi50C7dcoHXlc=
+modernc.org/opt v0.1.1 h1:/0RX92k9vwVeDXj+Xn23DKp2VJubL7k8qNffND6qn3A=
+modernc.org/opt v0.1.1/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
+modernc.org/sqlite v1.10.1-0.20210314190707-798bbeb9bb84 h1:rgEUzE849tFlHSoeCrKyS9cZAljC+DY7MdMHKq6R6sY=
+modernc.org/sqlite v1.10.1-0.20210314190707-798bbeb9bb84/go.mod h1:PGzq6qlhyYjL6uVbSgS6WoF7ZopTW/sI7+7p+mb4ZVU=
+modernc.org/strutil v1.1.0 h1:+1/yCzZxY2pZwwrsbH+4T7BQMoLQ9QiBshRC9eicYsc=
+modernc.org/strutil v1.1.0/go.mod h1:lstksw84oURvj9y3tn8lGvRxyRC1S2+g5uuIzNfIOBs=
+modernc.org/tcl v1.5.0 h1:euZSUNfE0Fd4W8VqXI1Ly1v7fqDJoBuAV88Ea+SnaSs=
+modernc.org/tcl v1.5.0/go.mod h1:gb57hj4pO8fRrK54zveIfFXBaMHK3SKJNWcmRw1cRzc=
+modernc.org/token v1.0.0 h1:a0jaWiNMDhDUtqOj09wvjWWAqd3q7WpBulmL9H2egsk=
+modernc.org/token v1.0.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+modernc.org/z v1.0.1-0.20210308123920-1f282aa71362/go.mod h1:8/SRk5C/HgiQWCgXdfpb+1RvhORdkz5sw72d3jjtyqA=
+modernc.org/z v1.0.1 h1:WyIDpEpAIx4Hel6q/Pcgj/VhaQV5XPJ2I6ryIYbjnpc=
+modernc.org/z v1.0.1/go.mod h1:8/SRk5C/HgiQWCgXdfpb+1RvhORdkz5sw72d3jjtyqA=
 mvdan.cc/xurls/v2 v2.2.0 h1:NSZPykBXJFCetGZykLAxaL6SIpvbVy/UFEniIfHAa8A=
 mvdan.cc/xurls/v2 v2.2.0/go.mod h1:EV1RMtya9D6G5DMYPGD8zTQzaHet6Jh8gFlRgGRJeO8=
 rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
@@ -1678,8 +1707,9 @@ sourcegraph.com/sourcegraph/appdash v0.0.0-20190731080439-ebfcffb1b5c0/go.mod h1
 strk.kbt.io/projects/go/libravatar v0.0.0-20191008002943-06d1c002b251 h1:mUcz5b3FJbP5Cvdq7Khzn6J9OCUQJaBwgBkCR+MOwSs=
 strk.kbt.io/projects/go/libravatar v0.0.0-20191008002943-06d1c002b251/go.mod h1:FJGmPh3vz9jSos1L/F91iAgnC/aejc0wIIrF2ZwJxdY=
 xorm.io/builder v0.3.7/go.mod h1:aUW0S9eb9VCaPohFCH3j7czOx1PMW3i1HrSzbLYGBSE=
+xorm.io/builder v0.3.8/go.mod h1:aUW0S9eb9VCaPohFCH3j7czOx1PMW3i1HrSzbLYGBSE=
 xorm.io/builder v0.3.9 h1:Sd65/LdWyO7LR8+Cbd+e7mm3sK/7U9k0jS3999IDHMc=
 xorm.io/builder v0.3.9/go.mod h1:aUW0S9eb9VCaPohFCH3j7czOx1PMW3i1HrSzbLYGBSE=
 xorm.io/xorm v1.0.6/go.mod h1:uF9EtbhODq5kNWxMbnBEj8hRRZnlcNSz2t2N7HW/+A4=
-xorm.io/xorm v1.0.7 h1:26yBTDVI+CfQpVz2Y88fISh+aiJXIPP4eNoTJlwzsC4=
-xorm.io/xorm v1.0.7/go.mod h1:uF9EtbhODq5kNWxMbnBEj8hRRZnlcNSz2t2N7HW/+A4=
+xorm.io/xorm v1.1.0 h1:mkEsQXLauZajiOld2cB2PkFcUZKePepPgs1bC1dw8RA=
+xorm.io/xorm v1.1.0/go.mod h1:EDzNHMuCVZNszkIRSLL2nI0zX+nQE8RstAVranlSfqI=

integrations/api_repo_test.go

@@ -223,7 +223,7 @@ func TestAPIViewRepo(t *testing.T) {
 	DecodeJSON(t, resp, &repo)
 	assert.EqualValues(t, 1, repo.ID)
 	assert.EqualValues(t, "repo1", repo.Name)
-	assert.EqualValues(t, 2, repo.Releases)
+	assert.EqualValues(t, 1, repo.Releases)
 	assert.EqualValues(t, 1, repo.OpenIssues)
 	assert.EqualValues(t, 3, repo.OpenPulls)
 

integrations/api_team_test.go

@@ -144,7 +144,9 @@ func TestAPITeamSearch(t *testing.T) {
 	var results TeamSearchResults
 
 	session := loginUser(t, user.Name)
+	csrf := GetCSRF(t, session, "/"+org.Name)
 	req := NewRequestf(t, "GET", "/api/v1/orgs/%s/teams/search?q=%s", org.Name, "_team")
+	req.Header.Add("X-Csrf-Token", csrf)
 	resp := session.MakeRequest(t, req, http.StatusOK)
 	DecodeJSON(t, resp, &results)
 	assert.NotEmpty(t, results.Data)
@@ -154,7 +156,9 @@ func TestAPITeamSearch(t *testing.T) {
 	// no access if not organization member
 	user5 := models.AssertExistsAndLoadBean(t, &models.User{ID: 5}).(*models.User)
 	session = loginUser(t, user5.Name)
+	csrf = GetCSRF(t, session, "/"+org.Name)
 	req = NewRequestf(t, "GET", "/api/v1/orgs/%s/teams/search?q=%s", org.Name, "team")
+	req.Header.Add("X-Csrf-Token", csrf)
 	resp = session.MakeRequest(t, req, http.StatusForbidden)
 
 }

integrations/git_smart_http_test.go

@@ -0,0 +1,69 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package integrations
+
+import (
+	"io/ioutil"
+	"net/http"
+	"net/url"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestGitSmartHTTP(t *testing.T) {
+	onGiteaRun(t, testGitSmartHTTP)
+}
+
+func testGitSmartHTTP(t *testing.T, u *url.URL) {
+	var kases = []struct {
+		p    string
+		code int
+	}{
+		{
+			p:    "user2/repo1/info/refs",
+			code: 200,
+		},
+		{
+			p:    "user2/repo1/HEAD",
+			code: 200,
+		},
+		{
+			p:    "user2/repo1/objects/info/alternates",
+			code: 404,
+		},
+		{
+			p:    "user2/repo1/objects/info/http-alternates",
+			code: 404,
+		},
+		{
+			p:    "user2/repo1/../../custom/conf/app.ini",
+			code: 404,
+		},
+		{
+			p:    "user2/repo1/objects/info/../../../../custom/conf/app.ini",
+			code: 404,
+		},
+		{
+			p:    `user2/repo1/objects/info/..\..\..\..\custom\conf\app.ini`,
+			code: 400,
+		},
+	}
+
+	for _, kase := range kases {
+		t.Run(kase.p, func(t *testing.T) {
+			p := u.String() + kase.p
+			req, err := http.NewRequest("GET", p, nil)
+			assert.NoError(t, err)
+			req.SetBasicAuth("user2", userPassword)
+			resp, err := http.DefaultClient.Do(req)
+			assert.NoError(t, err)
+			defer resp.Body.Close()
+			assert.EqualValues(t, kase.code, resp.StatusCode)
+			_, err = ioutil.ReadAll(resp.Body)
+			assert.NoError(t, err)
+		})
+	}
+}

integrations/goget_test.go

@@ -0,0 +1,35 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package integrations
+
+import (
+	"fmt"
+	"net/http"
+	"testing"
+
+	"code.gitea.io/gitea/modules/setting"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestGoGet(t *testing.T) {
+	defer prepareTestEnv(t)()
+
+	req := NewRequest(t, "GET", "/blah/glah/plah?go-get=1")
+	resp := MakeRequest(t, req, http.StatusOK)
+
+	expected := fmt.Sprintf(`<!doctype html>
+<html>
+	<head>
+		<meta name="go-import" content="%[1]s:%[2]s/blah/glah git %[3]sblah/glah.git">
+		<meta name="go-source" content="%[1]s:%[2]s/blah/glah _ %[3]sblah/glah/src/branch/master{/dir} %[3]sblah/glah/src/branch/master{/dir}/{file}#L{line}">
+	</head>
+	<body>
+		go get --insecure %[1]s:%[2]s/blah/glah
+	</body>
+</html>
+`, setting.Domain, setting.HTTPPort, setting.AppURL)
+
+	assert.Equal(t, expected, resp.Body.String())
+}

integrations/repo_branch_test.go

@@ -11,6 +11,7 @@ import (
 	"strings"
 	"testing"
 
+	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/modules/test"
 
 	"github.com/stretchr/testify/assert"
@@ -134,5 +135,13 @@ func TestCreateBranchInvalidCSRF(t *testing.T) {
 		"_csrf":           "fake_csrf",
 		"new_branch_name": "test",
 	})
-	session.MakeRequest(t, req, http.StatusBadRequest)
+	resp := session.MakeRequest(t, req, http.StatusFound)
+	loc := resp.Header().Get("Location")
+	assert.Equal(t, setting.AppSubURL+"/", loc)
+	resp = session.MakeRequest(t, NewRequest(t, "GET", loc), http.StatusOK)
+	htmlDoc := NewHTMLParser(t, resp.Body)
+	assert.Equal(t,
+		"Bad Request: Invalid CSRF token",
+		strings.TrimSpace(htmlDoc.doc.Find(".ui.message").Text()),
+	)
 }

models/consistency.go

@@ -141,6 +141,12 @@ func (milestone *Milestone) checkForConsistency(t *testing.T) {
 	actual := getCount(t, x.Where("is_closed=?", true), &Issue{MilestoneID: milestone.ID})
 	assert.EqualValues(t, milestone.NumClosedIssues, actual,
 		"Unexpected number of closed issues for milestone %+v", milestone)
+
+	completeness := 0
+	if milestone.NumIssues > 0 {
+		completeness = milestone.NumClosedIssues * 100 / milestone.NumIssues
+	}
+	assert.Equal(t, completeness, milestone.Completeness)
 }
 
 func (label *Label) checkForConsistency(t *testing.T) {

models/issue.go

@@ -648,8 +648,10 @@ func (issue *Issue) doChangeStatus(e *xorm.Session, doer *User, isMergePull bool
 	}
 
 	// Update issue count of milestone
-	if err := updateMilestoneClosedNum(e, issue.MilestoneID); err != nil {
-		return nil, err
+	if issue.MilestoneID > 0 {
+		if err := updateMilestoneCounters(e, issue.MilestoneID); err != nil {
+			return nil, err
+		}
 	}
 
 	if err := issue.updateClosedNum(e); err != nil {
@@ -912,7 +914,7 @@ func newIssue(e *xorm.Session, doer *User, opts NewIssueOptions) (err error) {
 	opts.Issue.Index = inserted.Index
 
 	if opts.Issue.MilestoneID > 0 {
-		if _, err = e.Exec("UPDATE `milestone` SET num_issues=num_issues+1 WHERE id=?", opts.Issue.MilestoneID); err != nil {
+		if err := updateMilestoneCounters(e, opts.Issue.MilestoneID); err != nil {
 			return err
 		}
 
@@ -1032,6 +1034,9 @@ func newIssueAttempt(repo *Repository, issue *Issue, labelIDs []int64, uuids []s
 
 // GetIssueByIndex returns raw issue without loading attributes by index in a repository.
 func GetIssueByIndex(repoID, index int64) (*Issue, error) {
+	if index < 1 {
+		return nil, ErrIssueNotExist{}
+	}
 	issue := &Issue{
 		RepoID: repoID,
 		Index:  index,
@@ -1086,7 +1091,7 @@ func getIssuesByIDs(e Engine, issueIDs []int64) ([]*Issue, error) {
 
 func getIssueIDsByRepoID(e Engine, repoID int64) ([]int64, error) {
 	ids := make([]int64, 0, 10)
-	err := e.Table("issue").Where("repo_id = ?", repoID).Find(&ids)
+	err := e.Table("issue").Cols("id").Where("repo_id = ?", repoID).Find(&ids)
 	return ids, err
 }
 

models/issue_milestone.go

@@ -129,8 +129,12 @@ func GetMilestoneByRepoIDANDName(repoID int64, name string) (*Milestone, error)
 
 // GetMilestoneByID returns the milestone via id .
 func GetMilestoneByID(id int64) (*Milestone, error) {
+	return getMilestoneByID(x, id)
+}
+
+func getMilestoneByID(e Engine, id int64) (*Milestone, error) {
 	var m Milestone
-	has, err := x.ID(id).Get(&m)
+	has, err := e.ID(id).Get(&m)
 	if err != nil {
 		return nil, err
 	} else if !has {
@@ -155,10 +159,6 @@ func UpdateMilestone(m *Milestone, oldIsClosed bool) error {
 		return err
 	}
 
-	if err := updateMilestoneCompleteness(sess, m.ID); err != nil {
-		return err
-	}
-
 	// if IsClosed changed, update milestone numbers of repository
 	if oldIsClosed != m.IsClosed {
 		if err := updateRepoMilestoneNum(sess, m.RepoID); err != nil {
@@ -171,23 +171,31 @@ func UpdateMilestone(m *Milestone, oldIsClosed bool) error {
 
 func updateMilestone(e Engine, m *Milestone) error {
 	m.Name = strings.TrimSpace(m.Name)
-	_, err := e.ID(m.ID).AllCols().
+	_, err := e.ID(m.ID).AllCols().Update(m)
+	if err != nil {
+		return err
+	}
+	return updateMilestoneCounters(e, m.ID)
+}
+
+// updateMilestoneCounters calculates NumIssues, NumClosesIssues and Completeness
+func updateMilestoneCounters(e Engine, id int64) error {
+	_, err := e.ID(id).
 		SetExpr("num_issues", builder.Select("count(*)").From("issue").Where(
-			builder.Eq{"milestone_id": m.ID},
+			builder.Eq{"milestone_id": id},
 		)).
 		SetExpr("num_closed_issues", builder.Select("count(*)").From("issue").Where(
 			builder.Eq{
-				"milestone_id": m.ID,
+				"milestone_id": id,
 				"is_closed":    true,
 			},
 		)).
-		Update(m)
-	return err
-}
-
-func updateMilestoneCompleteness(e Engine, milestoneID int64) error {
-	_, err := e.Exec("UPDATE `milestone` SET completeness=100*num_closed_issues/(CASE WHEN num_issues > 0 THEN num_issues ELSE 1 END) WHERE id=?",
-		milestoneID,
+		Update(&Milestone{})
+	if err != nil {
+		return err
+	}
+	_, err = e.Exec("UPDATE `milestone` SET completeness=100*num_closed_issues/(CASE WHEN num_issues > 0 THEN num_issues ELSE 1 END) WHERE id=?",
+		id,
 	)
 	return err
 }
@@ -256,25 +264,15 @@ func changeMilestoneAssign(e *xorm.Session, doer *User, issue *Issue, oldMilesto
 	}
 
 	if oldMilestoneID > 0 {
-		if err := updateMilestoneTotalNum(e, oldMilestoneID); err != nil {
+		if err := updateMilestoneCounters(e, oldMilestoneID); err != nil {
 			return err
 		}
-		if issue.IsClosed {
-			if err := updateMilestoneClosedNum(e, oldMilestoneID); err != nil {
-				return err
-			}
-		}
 	}
 
 	if issue.MilestoneID > 0 {
-		if err := updateMilestoneTotalNum(e, issue.MilestoneID); err != nil {
+		if err := updateMilestoneCounters(e, issue.MilestoneID); err != nil {
 			return err
 		}
-		if issue.IsClosed {
-			if err := updateMilestoneClosedNum(e, issue.MilestoneID); err != nil {
-				return err
-			}
-		}
 	}
 
 	if oldMilestoneID > 0 || issue.MilestoneID > 0 {
@@ -558,29 +556,6 @@ func updateRepoMilestoneNum(e Engine, repoID int64) error {
 	return err
 }
 
-func updateMilestoneTotalNum(e Engine, milestoneID int64) (err error) {
-	if _, err = e.Exec("UPDATE `milestone` SET num_issues=(SELECT count(*) FROM issue WHERE milestone_id=?) WHERE id=?",
-		milestoneID,
-		milestoneID,
-	); err != nil {
-		return
-	}
-
-	return updateMilestoneCompleteness(e, milestoneID)
-}
-
-func updateMilestoneClosedNum(e Engine, milestoneID int64) (err error) {
-	if _, err = e.Exec("UPDATE `milestone` SET num_closed_issues=(SELECT count(*) FROM issue WHERE milestone_id=? AND is_closed=?) WHERE id=?",
-		milestoneID,
-		true,
-		milestoneID,
-	); err != nil {
-		return
-	}
-
-	return updateMilestoneCompleteness(e, milestoneID)
-}
-
 //  _____               _            _ _____ _
 // |_   _| __ __ _  ___| | _____  __| |_   _(_)_ __ ___   ___  ___
 //   | || '__/ _` |/ __| |/ / _ \/ _` | | | | | '_ ` _ \ / _ \/ __|

models/issue_milestone_test.go

@@ -215,7 +215,7 @@ func TestChangeMilestoneStatus(t *testing.T) {
 	CheckConsistencyFor(t, &Repository{ID: milestone.RepoID}, &Milestone{})
 }
 
-func TestUpdateMilestoneClosedNum(t *testing.T) {
+func TestUpdateMilestoneCounters(t *testing.T) {
 	assert.NoError(t, PrepareTestDatabase())
 	issue := AssertExistsAndLoadBean(t, &Issue{MilestoneID: 1},
 		"is_closed=0").(*Issue)
@@ -224,14 +224,14 @@ func TestUpdateMilestoneClosedNum(t *testing.T) {
 	issue.ClosedUnix = timeutil.TimeStampNow()
 	_, err := x.ID(issue.ID).Cols("is_closed", "closed_unix").Update(issue)
 	assert.NoError(t, err)
-	assert.NoError(t, updateMilestoneClosedNum(x, issue.MilestoneID))
+	assert.NoError(t, updateMilestoneCounters(x, issue.MilestoneID))
 	CheckConsistencyFor(t, &Milestone{})
 
 	issue.IsClosed = false
 	issue.ClosedUnix = 0
 	_, err = x.ID(issue.ID).Cols("is_closed", "closed_unix").Update(issue)
 	assert.NoError(t, err)
-	assert.NoError(t, updateMilestoneClosedNum(x, issue.MilestoneID))
+	assert.NoError(t, updateMilestoneCounters(x, issue.MilestoneID))
 	CheckConsistencyFor(t, &Milestone{})
 }
 

models/issue_test.go

@@ -36,6 +36,14 @@ func TestIssue_ReplaceLabels(t *testing.T) {
 	testSuccess(1, []int64{})
 }
 
+func Test_GetIssueIDsByRepoID(t *testing.T) {
+	assert.NoError(t, PrepareTestDatabase())
+
+	ids, err := GetIssueIDsByRepoID(1)
+	assert.NoError(t, err)
+	assert.Len(t, ids, 5)
+}
+
 func TestIssueAPIURL(t *testing.T) {
 	assert.NoError(t, PrepareTestDatabase())
 	issue := AssertExistsAndLoadBean(t, &Issue{ID: 1}).(*Issue)

models/list_options.go

@@ -41,7 +41,7 @@ func (opts *ListOptions) setEnginePagination(e Engine) Engine {
 func (opts *ListOptions) GetStartEnd() (start, end int) {
 	opts.setDefaultValues()
 	start = (opts.Page - 1) * opts.PageSize
-	end = start + opts.Page
+	end = start + opts.PageSize
 	return
 }
 

models/login_source.go

@@ -21,6 +21,7 @@ import (
 	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/modules/timeutil"
 	"code.gitea.io/gitea/modules/util"
+	gouuid "github.com/google/uuid"
 	jsoniter "github.com/json-iterator/go"
 
 	"xorm.io/xorm"
@@ -68,6 +69,17 @@ var (
 	_ convert.Conversion = &SSPIConfig{}
 )
 
+// jsonUnmarshalIgnoreErroneousBOM - due to a bug in xorm (see https://gitea.com/xorm/xorm/pulls/1957) - it's
+// possible that a Blob may gain an unwanted prefix of 0xff 0xfe.
+func jsonUnmarshalIgnoreErroneousBOM(bs []byte, v interface{}) error {
+	json := jsoniter.ConfigCompatibleWithStandardLibrary
+	err := json.Unmarshal(bs, v)
+	if err != nil && len(bs) > 2 && bs[0] == 0xff && bs[1] == 0xfe {
+		err = json.Unmarshal(bs[2:], v)
+	}
+	return err
+}
+
 // LDAPConfig holds configuration for LDAP login source.
 type LDAPConfig struct {
 	*ldap.Source
@@ -75,8 +87,7 @@ type LDAPConfig struct {
 
 // FromDB fills up a LDAPConfig from serialized format.
 func (cfg *LDAPConfig) FromDB(bs []byte) error {
-	json := jsoniter.ConfigCompatibleWithStandardLibrary
-	return json.Unmarshal(bs, &cfg)
+	return jsonUnmarshalIgnoreErroneousBOM(bs, &cfg)
 }
 
 // ToDB exports a LDAPConfig to a serialized format.
@@ -103,8 +114,7 @@ type SMTPConfig struct {
 
 // FromDB fills up an SMTPConfig from serialized format.
 func (cfg *SMTPConfig) FromDB(bs []byte) error {
-	json := jsoniter.ConfigCompatibleWithStandardLibrary
-	return json.Unmarshal(bs, cfg)
+	return jsonUnmarshalIgnoreErroneousBOM(bs, cfg)
 }
 
 // ToDB exports an SMTPConfig to a serialized format.
@@ -116,12 +126,12 @@ func (cfg *SMTPConfig) ToDB() ([]byte, error) {
 // PAMConfig holds configuration for the PAM login source.
 type PAMConfig struct {
 	ServiceName string // pam service (e.g. system-auth)
+	EmailDomain string
 }
 
 // FromDB fills up a PAMConfig from serialized format.
 func (cfg *PAMConfig) FromDB(bs []byte) error {
-	json := jsoniter.ConfigCompatibleWithStandardLibrary
-	return json.Unmarshal(bs, &cfg)
+	return jsonUnmarshalIgnoreErroneousBOM(bs, cfg)
 }
 
 // ToDB exports a PAMConfig to a serialized format.
@@ -142,8 +152,7 @@ type OAuth2Config struct {
 
 // FromDB fills up an OAuth2Config from serialized format.
 func (cfg *OAuth2Config) FromDB(bs []byte) error {
-	json := jsoniter.ConfigCompatibleWithStandardLibrary
-	return json.Unmarshal(bs, cfg)
+	return jsonUnmarshalIgnoreErroneousBOM(bs, cfg)
 }
 
 // ToDB exports an SMTPConfig to a serialized format.
@@ -163,8 +172,7 @@ type SSPIConfig struct {
 
 // FromDB fills up an SSPIConfig from serialized format.
 func (cfg *SSPIConfig) FromDB(bs []byte) error {
-	json := jsoniter.ConfigCompatibleWithStandardLibrary
-	return json.Unmarshal(bs, cfg)
+	return jsonUnmarshalIgnoreErroneousBOM(bs, cfg)
 }
 
 // ToDB exports an SSPIConfig to a serialized format.
@@ -696,15 +704,26 @@ func LoginViaPAM(user *User, login, password string, sourceID int64, cfg *PAMCon
 
 	// Allow PAM sources with `@` in their name, like from Active Directory
 	username := pamLogin
+	email := pamLogin
 	idx := strings.Index(pamLogin, "@")
 	if idx > -1 {
 		username = pamLogin[:idx]
 	}
+	if ValidateEmail(email) != nil {
+		if cfg.EmailDomain != "" {
+			email = fmt.Sprintf("%s@%s", username, cfg.EmailDomain)
+		} else {
+			email = fmt.Sprintf("%s@%s", username, setting.Service.NoReplyAddress)
+		}
+		if ValidateEmail(email) != nil {
+			email = gouuid.New().String() + "@localhost"
+		}
+	}
 
 	user = &User{
 		LowerName:   strings.ToLower(username),
 		Name:        username,
-		Email:       pamLogin,
+		Email:       email,
 		Passwd:      password,
 		LoginType:   LoginPAM,
 		LoginSource: sourceID,

models/migrations/v156.go

@@ -88,7 +88,7 @@ func fixPublisherIDforTagReleases(x *xorm.Engine) error {
 				repo = new(Repository)
 				has, err := sess.ID(release.RepoID).Get(repo)
 				if err != nil {
-					log.Error("Error whilst loading repository[%d] for release[%d] with tag name %s", release.RepoID, release.ID, release.TagName)
+					log.Error("Error whilst loading repository[%d] for release[%d] with tag name %s. Error: %v", release.RepoID, release.ID, release.TagName, err)
 					return err
 				} else if !has {
 					log.Warn("Release[%d] is orphaned and refers to non-existing repository %d", release.ID, release.RepoID)
@@ -105,13 +105,13 @@ func fixPublisherIDforTagReleases(x *xorm.Engine) error {
 					}
 
 					if _, err := sess.ID(release.RepoID).Get(repo); err != nil {
-						log.Error("Error whilst loading repository[%d] for release[%d] with tag name %s", release.RepoID, release.ID, release.TagName)
+						log.Error("Error whilst loading repository[%d] for release[%d] with tag name %s. Error: %v", release.RepoID, release.ID, release.TagName, err)
 						return err
 					}
 				}
 				gitRepo, err = git.OpenRepository(repoPath(repo.OwnerName, repo.Name))
 				if err != nil {
-					log.Error("Error whilst opening git repo for %-v", repo)
+					log.Error("Error whilst opening git repo for [%d]%s/%s. Error: %v", repo.ID, repo.OwnerName, repo.Name, err)
 					return err
 				}
 			}
@@ -119,18 +119,36 @@ func fixPublisherIDforTagReleases(x *xorm.Engine) error {
 			commit, err := gitRepo.GetTagCommit(release.TagName)
 			if err != nil {
 				if git.IsErrNotExist(err) {
-					log.Warn("Unable to find commit %s for Tag: %s in %-v. Cannot update publisher ID.", err.(git.ErrNotExist).ID, release.TagName, repo)
+					log.Warn("Unable to find commit %s for Tag: %s in [%d]%s/%s. Cannot update publisher ID.", err.(git.ErrNotExist).ID, release.TagName, repo.ID, repo.OwnerName, repo.Name)
 					continue
 				}
-				log.Error("Error whilst getting commit for Tag: %s in %-v.", release.TagName, repo)
+				log.Error("Error whilst getting commit for Tag: %s in [%d]%s/%s. Error: %v", release.TagName, repo.ID, repo.OwnerName, repo.Name, err)
 				return fmt.Errorf("GetTagCommit: %v", err)
 			}
 
+			if commit.Author.Email == "" {
+				log.Warn("Tag: %s in Repo[%d]%s/%s does not have a tagger.", release.TagName, repo.ID, repo.OwnerName, repo.Name)
+				commit, err = gitRepo.GetCommit(commit.ID.String())
+				if err != nil {
+					if git.IsErrNotExist(err) {
+						log.Warn("Unable to find commit %s for Tag: %s in [%d]%s/%s. Cannot update publisher ID.", err.(git.ErrNotExist).ID, release.TagName, repo.ID, repo.OwnerName, repo.Name)
+						continue
+					}
+					log.Error("Error whilst getting commit for Tag: %s in [%d]%s/%s. Error: %v", release.TagName, repo.ID, repo.OwnerName, repo.Name, err)
+					return fmt.Errorf("GetCommit: %v", err)
+				}
+			}
+
+			if commit.Author.Email == "" {
+				log.Warn("Tag: %s in Repo[%d]%s/%s does not have a Tagger and its underlying commit does not have an Author either!", release.TagName, repo.ID, repo.OwnerName, repo.Name)
+				continue
+			}
+
 			if user == nil || !strings.EqualFold(user.Email, commit.Author.Email) {
 				user = new(User)
 				_, err = sess.Where("email=?", commit.Author.Email).Get(user)
 				if err != nil {
-					log.Error("Error whilst getting commit author by email: %s for Tag: %s in %-v.", commit.Author.Email, release.TagName, repo)
+					log.Error("Error whilst getting commit author by email: %s for Tag: %s in [%d]%s/%s. Error: %v", commit.Author.Email, release.TagName, repo.ID, repo.OwnerName, repo.Name, err)
 					return err
 				}
 
@@ -143,7 +161,7 @@ func fixPublisherIDforTagReleases(x *xorm.Engine) error {
 
 			release.PublisherID = user.ID
 			if _, err := sess.ID(release.ID).Cols("publisher_id").Update(release); err != nil {
-				log.Error("Error whilst updating publisher[%d] for release[%d] with tag name %s", release.PublisherID, release.ID, release.TagName)
+				log.Error("Error whilst updating publisher[%d] for release[%d] with tag name %s. Error: %v", release.PublisherID, release.ID, release.TagName, err)
 				return err
 			}
 		}

models/repo.go

@@ -1215,7 +1215,7 @@ func ChangeRepositoryName(doer *User, repo *Repository, newRepoName string) (err
 	}
 
 	newRepoPath := RepoPath(repo.Owner.Name, newRepoName)
-	if err = os.Rename(repo.RepoPath(), newRepoPath); err != nil {
+	if err = util.Rename(repo.RepoPath(), newRepoPath); err != nil {
 		return fmt.Errorf("rename repository directory: %v", err)
 	}
 
@@ -1226,7 +1226,7 @@ func ChangeRepositoryName(doer *User, repo *Repository, newRepoName string) (err
 		return err
 	}
 	if isExist {
-		if err = os.Rename(wikiPath, WikiPath(repo.Owner.Name, newRepoName)); err != nil {
+		if err = util.Rename(wikiPath, WikiPath(repo.Owner.Name, newRepoName)); err != nil {
 			return fmt.Errorf("rename repository wiki: %v", err)
 		}
 	}
@@ -1349,6 +1349,26 @@ func UpdateRepository(repo *Repository, visibilityChanged bool) (err error) {
 	return sess.Commit()
 }
 
+// UpdateRepositoryOwnerNames updates repository owner_names (this should only be used when the ownerName has changed case)
+func UpdateRepositoryOwnerNames(ownerID int64, ownerName string) error {
+	if ownerID == 0 {
+		return nil
+	}
+	sess := x.NewSession()
+	defer sess.Close()
+	if err := sess.Begin(); err != nil {
+		return err
+	}
+
+	if _, err := sess.Where("owner_id = ?", ownerID).Cols("owner_name").Update(&Repository{
+		OwnerName: ownerName,
+	}); err != nil {
+		return err
+	}
+
+	return sess.Commit()
+}
+
 // UpdateRepositoryUpdatedTime updates a repository's updated time
 func UpdateRepositoryUpdatedTime(repoID int64, updateTime time.Time) error {
 	_, err := x.Exec("UPDATE repository SET updated_unix = ? WHERE id = ?", updateTime.Unix(), repoID)

models/repo_transfer.go

@@ -210,13 +210,13 @@ func TransferOwnership(doer *User, newOwnerName string, repo *Repository) (err e
 		}
 
 		if repoRenamed {
-			if err := os.Rename(RepoPath(newOwnerName, repo.Name), RepoPath(oldOwnerName, repo.Name)); err != nil {
+			if err := util.Rename(RepoPath(newOwnerName, repo.Name), RepoPath(oldOwnerName, repo.Name)); err != nil {
 				log.Critical("Unable to move repository %s/%s directory from %s back to correct place %s: %v", oldOwnerName, repo.Name, RepoPath(newOwnerName, repo.Name), RepoPath(oldOwnerName, repo.Name), err)
 			}
 		}
 
 		if wikiRenamed {
-			if err := os.Rename(WikiPath(newOwnerName, repo.Name), WikiPath(oldOwnerName, repo.Name)); err != nil {
+			if err := util.Rename(WikiPath(newOwnerName, repo.Name), WikiPath(oldOwnerName, repo.Name)); err != nil {
 				log.Critical("Unable to move wiki for repository %s/%s directory from %s back to correct place %s: %v", oldOwnerName, repo.Name, WikiPath(newOwnerName, repo.Name), WikiPath(oldOwnerName, repo.Name), err)
 			}
 		}
@@ -358,7 +358,7 @@ func TransferOwnership(doer *User, newOwnerName string, repo *Repository) (err e
 		return fmt.Errorf("Failed to create dir %s: %v", dir, err)
 	}
 
-	if err := os.Rename(RepoPath(oldOwner.Name, repo.Name), RepoPath(newOwner.Name, repo.Name)); err != nil {
+	if err := util.Rename(RepoPath(oldOwner.Name, repo.Name), RepoPath(newOwner.Name, repo.Name)); err != nil {
 		return fmt.Errorf("rename repository directory: %v", err)
 	}
 	repoRenamed = true
@@ -370,7 +370,7 @@ func TransferOwnership(doer *User, newOwnerName string, repo *Repository) (err e
 		log.Error("Unable to check if %s exists. Error: %v", wikiPath, err)
 		return err
 	} else if isExist {
-		if err := os.Rename(wikiPath, WikiPath(newOwner.Name, repo.Name)); err != nil {
+		if err := util.Rename(wikiPath, WikiPath(newOwner.Name, repo.Name)); err != nil {
 			return fmt.Errorf("rename repository wiki: %v", err)
 		}
 		wikiRenamed = true

models/repo_unit.go

@@ -28,8 +28,7 @@ type UnitConfig struct{}
 
 // FromDB fills up a UnitConfig from serialized format.
 func (cfg *UnitConfig) FromDB(bs []byte) error {
-	json := jsoniter.ConfigCompatibleWithStandardLibrary
-	return json.Unmarshal(bs, &cfg)
+	return jsonUnmarshalIgnoreErroneousBOM(bs, &cfg)
 }
 
 // ToDB exports a UnitConfig to a serialized format.
@@ -45,8 +44,7 @@ type ExternalWikiConfig struct {
 
 // FromDB fills up a ExternalWikiConfig from serialized format.
 func (cfg *ExternalWikiConfig) FromDB(bs []byte) error {
-	json := jsoniter.ConfigCompatibleWithStandardLibrary
-	return json.Unmarshal(bs, &cfg)
+	return jsonUnmarshalIgnoreErroneousBOM(bs, &cfg)
 }
 
 // ToDB exports a ExternalWikiConfig to a serialized format.
@@ -64,8 +62,7 @@ type ExternalTrackerConfig struct {
 
 // FromDB fills up a ExternalTrackerConfig from serialized format.
 func (cfg *ExternalTrackerConfig) FromDB(bs []byte) error {
-	json := jsoniter.ConfigCompatibleWithStandardLibrary
-	return json.Unmarshal(bs, &cfg)
+	return jsonUnmarshalIgnoreErroneousBOM(bs, &cfg)
 }
 
 // ToDB exports a ExternalTrackerConfig to a serialized format.
@@ -83,8 +80,7 @@ type IssuesConfig struct {
 
 // FromDB fills up a IssuesConfig from serialized format.
 func (cfg *IssuesConfig) FromDB(bs []byte) error {
-	json := jsoniter.ConfigCompatibleWithStandardLibrary
-	return json.Unmarshal(bs, &cfg)
+	return jsonUnmarshalIgnoreErroneousBOM(bs, &cfg)
 }
 
 // ToDB exports a IssuesConfig to a serialized format.
@@ -106,8 +102,7 @@ type PullRequestsConfig struct {
 
 // FromDB fills up a PullRequestsConfig from serialized format.
 func (cfg *PullRequestsConfig) FromDB(bs []byte) error {
-	json := jsoniter.ConfigCompatibleWithStandardLibrary
-	return json.Unmarshal(bs, &cfg)
+	return jsonUnmarshalIgnoreErroneousBOM(bs, &cfg)
 }
 
 // ToDB exports a PullRequestsConfig to a serialized format.

models/ssh_key.go

@@ -834,7 +834,7 @@ func rewriteAllPublicKeys(e Engine) error {
 	}
 
 	t.Close()
-	return os.Rename(tmpPath, fPath)
+	return util.Rename(tmpPath, fPath)
 }
 
 // RegeneratePublicKeys regenerates the authorized_keys file
@@ -1316,7 +1316,7 @@ func rewriteAllPrincipalKeys(e Engine) error {
 	}
 
 	t.Close()
-	return os.Rename(tmpPath, fPath)
+	return util.Rename(tmpPath, fPath)
 }
 
 // ListPrincipalKeys returns a list of principals belongs to given user.

models/task.go

@@ -8,8 +8,11 @@ import (
 	"fmt"
 
 	migration "code.gitea.io/gitea/modules/migrations/base"
+	"code.gitea.io/gitea/modules/secret"
+	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/modules/structs"
 	"code.gitea.io/gitea/modules/timeutil"
+	"code.gitea.io/gitea/modules/util"
 	jsoniter "github.com/json-iterator/go"
 
 	"xorm.io/builder"
@@ -110,6 +113,24 @@ func (task *Task) MigrateConfig() (*migration.MigrateOptions, error) {
 		if err != nil {
 			return nil, err
 		}
+
+		// decrypt credentials
+		if opts.CloneAddrEncrypted != "" {
+			if opts.CloneAddr, err = secret.DecryptSecret(setting.SecretKey, opts.CloneAddrEncrypted); err != nil {
+				return nil, err
+			}
+		}
+		if opts.AuthPasswordEncrypted != "" {
+			if opts.AuthPassword, err = secret.DecryptSecret(setting.SecretKey, opts.AuthPasswordEncrypted); err != nil {
+				return nil, err
+			}
+		}
+		if opts.AuthTokenEncrypted != "" {
+			if opts.AuthToken, err = secret.DecryptSecret(setting.SecretKey, opts.AuthTokenEncrypted); err != nil {
+				return nil, err
+			}
+		}
+
 		return &opts, nil
 	}
 	return nil, fmt.Errorf("Task type is %s, not Migrate Repo", task.Type.Name())
@@ -205,12 +226,31 @@ func createTask(e Engine, task *Task) error {
 func FinishMigrateTask(task *Task) error {
 	task.Status = structs.TaskStatusFinished
 	task.EndTime = timeutil.TimeStampNow()
+
+	// delete credentials when we're done, they're a liability.
+	conf, err := task.MigrateConfig()
+	if err != nil {
+		return err
+	}
+	conf.AuthPassword = ""
+	conf.AuthToken = ""
+	conf.CloneAddr = util.SanitizeURLCredentials(conf.CloneAddr, true)
+	conf.AuthPasswordEncrypted = ""
+	conf.AuthTokenEncrypted = ""
+	conf.CloneAddrEncrypted = ""
+	json := jsoniter.ConfigCompatibleWithStandardLibrary
+	confBytes, err := json.Marshal(conf)
+	if err != nil {
+		return err
+	}
+	task.PayloadContent = string(confBytes)
+
 	sess := x.NewSession()
 	defer sess.Close()
 	if err := sess.Begin(); err != nil {
 		return err
 	}
-	if _, err := sess.ID(task.ID).Cols("status", "end_time").Update(task); err != nil {
+	if _, err := sess.ID(task.ID).Cols("status", "end_time", "payload_content").Update(task); err != nil {
 		return err
 	}
 

models/token.go

@@ -57,9 +57,15 @@ func GetAccessTokenBySHA(token string) (*AccessToken, error) {
 	if token == "" {
 		return nil, ErrAccessTokenEmpty{}
 	}
-	if len(token) < 8 {
+	// A token is defined as being SHA1 sum these are 40 hexadecimal bytes long
+	if len(token) != 40 {
 		return nil, ErrAccessTokenNotExist{token}
 	}
+	for _, x := range []byte(token) {
+		if x < '0' || (x > '9' && x < 'a') || x > 'f' {
+			return nil, ErrAccessTokenNotExist{token}
+		}
+	}
 	var tokens []AccessToken
 	lastEight := token[len(token)-8:]
 	err := x.Table(&AccessToken{}).Where("token_last_eight = ?", lastEight).Find(&tokens)

models/user.go

@@ -1011,7 +1011,7 @@ func ChangeUserName(u *User, newUserName string) (err error) {
 	}
 
 	// Do not fail if directory does not exist
-	if err = os.Rename(UserPath(oldUserName), UserPath(newUserName)); err != nil && !os.IsNotExist(err) {
+	if err = util.Rename(UserPath(oldUserName), UserPath(newUserName)); err != nil && !os.IsNotExist(err) {
 		return fmt.Errorf("Rename user directory: %v", err)
 	}
 
@@ -1020,7 +1020,7 @@ func ChangeUserName(u *User, newUserName string) (err error) {
 	}
 
 	if err = sess.Commit(); err != nil {
-		if err2 := os.Rename(UserPath(newUserName), UserPath(oldUserName)); err2 != nil && !os.IsNotExist(err2) {
+		if err2 := util.Rename(UserPath(newUserName), UserPath(oldUserName)); err2 != nil && !os.IsNotExist(err2) {
 			log.Critical("Unable to rollback directory change during failed username change from: %s to: %s. DB Error: %v. Filesystem Error: %v", oldUserName, newUserName, err, err2)
 			return fmt.Errorf("failed to rollback directory change during failed username change from: %s to: %s. DB Error: %w. Filesystem Error: %v", oldUserName, newUserName, err, err2)
 		}

modules/base/tool.go

@@ -21,6 +21,7 @@ import (
 	"strings"
 	"time"
 	"unicode"
+	"unicode/utf8"
 
 	"code.gitea.io/gitea/modules/git"
 	"code.gitea.io/gitea/modules/log"
@@ -213,19 +214,19 @@ func EllipsisString(str string, length int) string {
 	if length <= 3 {
 		return "..."
 	}
-	if len(str) <= length {
+	if utf8.RuneCountInString(str) <= length {
 		return str
 	}
-	return str[:length-3] + "..."
+	return string([]rune(str)[:length-3]) + "..."
 }
 
 // TruncateString returns a truncated string with given limit,
 // it returns input string if length is not reached limit.
 func TruncateString(str string, limit int) string {
-	if len(str) < limit {
+	if utf8.RuneCountInString(str) < limit {
 		return str
 	}
-	return str[:limit]
+	return string([]rune(str)[:limit])
 }
 
 // StringsToInt64s converts a slice of string to a slice of int64.

modules/base/tool_test.go

@@ -170,6 +170,10 @@ func TestEllipsisString(t *testing.T) {
 	assert.Equal(t, "fo...", EllipsisString("foobar", 5))
 	assert.Equal(t, "foobar", EllipsisString("foobar", 6))
 	assert.Equal(t, "foobar", EllipsisString("foobar", 10))
+	assert.Equal(t, "测...", EllipsisString("测试文本一二三四", 4))
+	assert.Equal(t, "测试...", EllipsisString("测试文本一二三四", 5))
+	assert.Equal(t, "测试文...", EllipsisString("测试文本一二三四", 6))
+	assert.Equal(t, "测试文本一二三四", EllipsisString("测试文本一二三四", 10))
 }
 
 func TestTruncateString(t *testing.T) {
@@ -181,6 +185,10 @@ func TestTruncateString(t *testing.T) {
 	assert.Equal(t, "fooba", TruncateString("foobar", 5))
 	assert.Equal(t, "foobar", TruncateString("foobar", 6))
 	assert.Equal(t, "foobar", TruncateString("foobar", 7))
+	assert.Equal(t, "测试文本", TruncateString("测试文本一二三四", 4))
+	assert.Equal(t, "测试文本一", TruncateString("测试文本一二三四", 5))
+	assert.Equal(t, "测试文本一二", TruncateString("测试文本一二三四", 6))
+	assert.Equal(t, "测试文本一二三", TruncateString("测试文本一二三四", 7))
 }
 
 func TestStringsToInt64s(t *testing.T) {

modules/context/csrf.go

@@ -22,6 +22,7 @@ import (
 	"net/http"
 	"time"
 
+	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/modules/web/middleware"
 
 	"github.com/unknwon/com"
@@ -266,7 +267,12 @@ func Validate(ctx *Context, x CSRF) {
 				-1,
 				x.GetCookiePath(),
 				x.GetCookieDomain()) // FIXME: Do we need to set the Secure, httpOnly and SameSite values too?
-			x.Error(ctx.Resp)
+			if middleware.IsAPIPath(ctx.Req) {
+				x.Error(ctx.Resp)
+				return
+			}
+			ctx.Flash.Error(ctx.Tr("error.invalid_csrf"))
+			ctx.Redirect(setting.AppSubURL + "/")
 		}
 		return
 	}
@@ -277,10 +283,19 @@ func Validate(ctx *Context, x CSRF) {
 				-1,
 				x.GetCookiePath(),
 				x.GetCookieDomain()) // FIXME: Do we need to set the Secure, httpOnly and SameSite values too?
-			x.Error(ctx.Resp)
+			if middleware.IsAPIPath(ctx.Req) {
+				x.Error(ctx.Resp)
+				return
+			}
+			ctx.Flash.Error(ctx.Tr("error.invalid_csrf"))
+			ctx.Redirect(setting.AppSubURL + "/")
 		}
 		return
 	}
-
-	http.Error(ctx.Resp, "Bad Request: no CSRF token present", http.StatusBadRequest)
+	if middleware.IsAPIPath(ctx.Req) {
+		http.Error(ctx.Resp, "Bad Request: no CSRF token present", http.StatusBadRequest)
+		return
+	}
+	ctx.Flash.Error(ctx.Tr("error.missing_csrf"))
+	ctx.Redirect(setting.AppSubURL + "/")
 }

modules/context/response.go

@@ -49,7 +49,7 @@ func (r *Response) Write(bs []byte) (int, error) {
 		return size, err
 	}
 	if r.status == 0 {
-		r.WriteHeader(200)
+		r.status = http.StatusOK
 	}
 	return size, nil
 }

modules/convert/git_commit.go

@@ -155,8 +155,8 @@ func ToCommit(repo *models.Repository, commit *git.Commit, userCache map[string]
 			URL: repo.APIURL() + "/git/commits/" + commit.ID.String(),
 			Author: &api.CommitUser{
 				Identity: api.Identity{
-					Name:  commit.Committer.Name,
-					Email: commit.Committer.Email,
+					Name:  commit.Author.Name,
+					Email: commit.Author.Email,
 				},
 				Date: commit.Author.When.Format(time.RFC3339),
 			},

modules/convert/repository.go

@@ -89,7 +89,7 @@ func innerToRepo(repo *models.Repository, mode models.AccessMode, isParent bool)
 		return nil
 	}
 
-	numReleases, _ := models.GetReleaseCountByRepoID(repo.ID, models.FindReleasesOptions{IncludeDrafts: false, IncludeTags: true})
+	numReleases, _ := models.GetReleaseCountByRepoID(repo.ID, models.FindReleasesOptions{IncludeDrafts: false, IncludeTags: false})
 
 	mirrorInterval := ""
 	if repo.IsMirror {

modules/doctor/dbversion.go

@@ -23,7 +23,7 @@ func checkDBVersion(logger log.Logger, autofix bool) error {
 
 		err = models.NewEngine(context.Background(), migrations.Migrate)
 		if err != nil {
-			logger.Critical("Error: %v during migration")
+			logger.Critical("Error: %v during migration", err)
 		}
 		return err
 	}

modules/emoji/emoji.go

@@ -6,6 +6,7 @@
 package emoji
 
 import (
+	"io"
 	"sort"
 	"strings"
 	"sync"
@@ -145,6 +146,8 @@ func (n *rememberSecondWriteWriter) Write(p []byte) (int, error) {
 	if n.writecount == 2 {
 		n.idx = n.pos
 		n.end = n.pos + len(p)
+		n.pos += len(p)
+		return len(p), io.EOF
 	}
 	n.pos += len(p)
 	return len(p), nil
@@ -155,6 +158,8 @@ func (n *rememberSecondWriteWriter) WriteString(s string) (int, error) {
 	if n.writecount == 2 {
 		n.idx = n.pos
 		n.end = n.pos + len(s)
+		n.pos += len(s)
+		return len(s), io.EOF
 	}
 	n.pos += len(s)
 	return len(s), nil

modules/forms/auth_form.go

@@ -51,6 +51,7 @@ type AuthenticationForm struct {
 	TLS                           bool
 	SkipVerify                    bool
 	PAMServiceName                string
+	PAMEmailDomain                string
 	Oauth2Provider                string
 	Oauth2Key                     string
 	Oauth2Secret                  string

modules/git/batch_reader.go

@@ -149,17 +149,18 @@ headerLoop:
 // constant hextable to help quickly convert between 20byte and 40byte hashes
 const hextable = "0123456789abcdef"
 
-// To40ByteSHA converts a 20-byte SHA in a 40-byte slice into a 40-byte sha in place
-// without allocations. This is at least 100x quicker that hex.EncodeToString
-// NB This requires that sha is a 40-byte slice
-func To40ByteSHA(sha []byte) []byte {
+// To40ByteSHA converts a 20-byte SHA into a 40-byte sha. Input and output can be the
+// same 40 byte slice to support in place conversion without allocations.
+// This is at least 100x quicker that hex.EncodeToString
+// NB This requires that out is a 40-byte slice
+func To40ByteSHA(sha, out []byte) []byte {
 	for i := 19; i >= 0; i-- {
 		v := sha[i]
 		vhi, vlo := v>>4, v&0x0f
 		shi, slo := hextable[vhi], hextable[vlo]
-		sha[i*2], sha[i*2+1] = shi, slo
+		out[i*2], out[i*2+1] = shi, slo
 	}
-	return sha
+	return out
 }
 
 // ParseTreeLineSkipMode reads an entry from a tree in a cat-file --batch stream

modules/git/command.go

@@ -124,12 +124,18 @@ func (c *Command) RunInDirTimeoutEnvFullPipelineFunc(env []string, timeout time.
 
 	cmd := exec.CommandContext(ctx, c.name, c.args...)
 	if env == nil {
-		cmd.Env = append(os.Environ(), fmt.Sprintf("LC_ALL=%s", DefaultLocale))
+		cmd.Env = os.Environ()
 	} else {
 		cmd.Env = env
-		cmd.Env = append(cmd.Env, fmt.Sprintf("LC_ALL=%s", DefaultLocale))
 	}
 
+	cmd.Env = append(
+		cmd.Env,
+		fmt.Sprintf("LC_ALL=%s", DefaultLocale),
+		// avoid prompting for credentials interactively, supported since git v2.3
+		"GIT_TERMINAL_PROMPT=0",
+	)
+
 	// TODO: verify if this is still needed in golang 1.15
 	if goVersionLessThan115 {
 		cmd.Env = append(cmd.Env, "GODEBUG=asyncpreemptoff=1")

modules/git/commit_info_nogogit.go

@@ -303,7 +303,7 @@ revListLoop:
 					commits[0] = string(commitID)
 				}
 			}
-			treeID = To40ByteSHA(treeID)
+			treeID = To40ByteSHA(treeID, treeID)
 			_, err = batchStdinWriter.Write(treeID)
 			if err != nil {
 				return nil, err

modules/git/commit_reader.go

@@ -17,7 +17,9 @@ import (
 // If used as part of a cat-file --batch stream you need to limit the reader to the correct size
 func CommitFromReader(gitRepo *Repository, sha SHA1, reader io.Reader) (*Commit, error) {
 	commit := &Commit{
-		ID: sha,
+		ID:        sha,
+		Author:    &Signature{},
+		Committer: &Signature{},
 	}
 
 	payloadSB := new(strings.Builder)

modules/git/pipeline/lfs_nogogit.go

@@ -43,8 +43,6 @@ func FindLFSFile(repo *git.Repository, hash git.SHA1) ([]*LFSResult, error) {
 
 	basePath := repo.Path
 
-	hashStr := hash.String()
-
 	// Use rev-list to provide us with all commits in order
 	revListReader, revListWriter := io.Pipe()
 	defer func() {
@@ -74,7 +72,7 @@ func FindLFSFile(repo *git.Repository, hash git.SHA1) ([]*LFSResult, error) {
 
 	fnameBuf := make([]byte, 4096)
 	modeBuf := make([]byte, 40)
-	workingShaBuf := make([]byte, 40)
+	workingShaBuf := make([]byte, 20)
 
 	for scan.Scan() {
 		// Get the next commit ID
@@ -132,8 +130,7 @@ func FindLFSFile(repo *git.Repository, hash git.SHA1) ([]*LFSResult, error) {
 						return nil, err
 					}
 					n += int64(count)
-					sha := git.To40ByteSHA(sha20byte)
-					if bytes.Equal(sha, []byte(hashStr)) {
+					if bytes.Equal(sha20byte, hash[:]) {
 						result := LFSResult{
 							Name:         curPath + string(fname),
 							SHA:          curCommit.ID.String(),
@@ -143,7 +140,9 @@ func FindLFSFile(repo *git.Repository, hash git.SHA1) ([]*LFSResult, error) {
 						}
 						resultsMap[curCommit.ID.String()+":"+curPath+string(fname)] = &result
 					} else if string(mode) == git.EntryModeTree.String() {
-						trees = append(trees, sha)
+						sha40Byte := make([]byte, 40)
+						git.To40ByteSHA(sha20byte, sha40Byte)
+						trees = append(trees, sha40Byte)
 						paths = append(paths, curPath+string(fname)+"/")
 					}
 				}

modules/git/repo_test.go

@@ -7,23 +7,18 @@ package git
 import (
 	"path/filepath"
 	"testing"
-	"time"
 
 	"github.com/stretchr/testify/assert"
 )
 
 func TestGetLatestCommitTime(t *testing.T) {
-	lct, err := GetLatestCommitTime(".")
+	bareRepo1Path := filepath.Join(testReposDir, "repo1_bare")
+	lct, err := GetLatestCommitTime(bareRepo1Path)
 	assert.NoError(t, err)
-	// Time is in the past
-	now := time.Now()
-	assert.True(t, lct.Unix() < now.Unix(), "%d not smaller than %d", lct, now)
-	// Time is after Mon Oct 23 03:52:09 2017 +0300
+	// Time is Sun Jul 21 22:43:13 2019 +0200
 	// which is the time of commit
-	// d47b98c44c9a6472e44ab80efe65235e11c6da2a
-	refTime, err := time.Parse("Mon Jan 02 15:04:05 2006 -0700", "Mon Oct 23 03:52:09 2017 +0300")
-	assert.NoError(t, err)
-	assert.True(t, lct.Unix() > refTime.Unix(), "%d not greater than %d", lct, refTime)
+	// feaf4ba6bc635fec442f46ddd4512416ec43c2c2 (refs/heads/master)
+	assert.EqualValues(t, 1563741793, lct.Unix())
 }
 
 func TestRepoIsEmpty(t *testing.T) {

modules/git/tag.go

@@ -35,6 +35,7 @@ func (tag *Tag) Commit() (*Commit, error) {
 // \n\n separate headers from message
 func parseTagData(data []byte) (*Tag, error) {
 	tag := new(Tag)
+	tag.Tagger = &Signature{}
 	// we now have the contents of the commit object. Let's investigate...
 	nextline := 0
 l:

modules/graceful/server.go

@@ -17,6 +17,7 @@ import (
 	"time"
 
 	"code.gitea.io/gitea/modules/log"
+	"code.gitea.io/gitea/modules/setting"
 )
 
 var (
@@ -26,6 +27,10 @@ var (
 	DefaultWriteTimeOut time.Duration
 	// DefaultMaxHeaderBytes default max header bytes
 	DefaultMaxHeaderBytes int
+	// PerWriteWriteTimeout timeout for writes
+	PerWriteWriteTimeout = 30 * time.Second
+	// PerWriteWriteTimeoutKbTime is a timeout taking account of how much there is to be written
+	PerWriteWriteTimeoutKbTime = 10 * time.Second
 )
 
 func init() {
@@ -37,14 +42,16 @@ type ServeFunction = func(net.Listener) error
 
 // Server represents our graceful server
 type Server struct {
-	network     string
-	address     string
-	listener    net.Listener
-	wg          sync.WaitGroup
-	state       state
-	lock        *sync.RWMutex
-	BeforeBegin func(network, address string)
-	OnShutdown  func()
+	network              string
+	address              string
+	listener             net.Listener
+	wg                   sync.WaitGroup
+	state                state
+	lock                 *sync.RWMutex
+	BeforeBegin          func(network, address string)
+	OnShutdown           func()
+	PerWriteTimeout      time.Duration
+	PerWritePerKbTimeout time.Duration
 }
 
 // NewServer creates a server on network at provided address
@@ -55,11 +62,13 @@ func NewServer(network, address, name string) *Server {
 		log.Info("Starting new %s server: %s:%s on PID: %d", name, network, address, os.Getpid())
 	}
 	srv := &Server{
-		wg:      sync.WaitGroup{},
-		state:   stateInit,
-		lock:    &sync.RWMutex{},
-		network: network,
-		address: address,
+		wg:                   sync.WaitGroup{},
+		state:                stateInit,
+		lock:                 &sync.RWMutex{},
+		network:              network,
+		address:              address,
+		PerWriteTimeout:      setting.PerWriteTimeout,
+		PerWritePerKbTimeout: setting.PerWritePerKbTimeout,
 	}
 
 	srv.BeforeBegin = func(network, addr string) {
@@ -221,9 +230,11 @@ func (wl *wrappedListener) Accept() (net.Conn, error) {
 	closed := int32(0)
 
 	c = wrappedConn{
-		Conn:   c,
-		server: wl.server,
-		closed: &closed,
+		Conn:                 c,
+		server:               wl.server,
+		closed:               &closed,
+		perWriteTimeout:      wl.server.PerWriteTimeout,
+		perWritePerKbTimeout: wl.server.PerWritePerKbTimeout,
 	}
 
 	wl.server.wg.Add(1)
@@ -246,8 +257,25 @@ func (wl *wrappedListener) File() (*os.File, error) {
 
 type wrappedConn struct {
 	net.Conn
-	server *Server
-	closed *int32
+	server               *Server
+	closed               *int32
+	deadline             time.Time
+	perWriteTimeout      time.Duration
+	perWritePerKbTimeout time.Duration
+}
+
+func (w wrappedConn) Write(p []byte) (n int, err error) {
+	if w.perWriteTimeout > 0 {
+		minTimeout := time.Duration(len(p)/1024) * w.perWritePerKbTimeout
+		minDeadline := time.Now().Add(minTimeout).Add(w.perWriteTimeout)
+
+		w.deadline = w.deadline.Add(minTimeout)
+		if minDeadline.After(w.deadline) {
+			w.deadline = minDeadline
+		}
+		_ = w.Conn.SetWriteDeadline(w.deadline)
+	}
+	return w.Conn.Write(p)
 }
 
 func (w wrappedConn) Close() error {

modules/log/file.go

@@ -177,7 +177,7 @@ func (log *FileLogger) DoRotate() error {
 
 		// close fd before rename
 		// Rename the file to its newfound home
-		if err = os.Rename(log.Filename, fname); err != nil {
+		if err = util.Rename(log.Filename, fname); err != nil {
 			return fmt.Errorf("Rotate: %v", err)
 		}
 

modules/markup/html.go

@@ -87,6 +87,7 @@ func isLinkStr(link string) bool {
 	return validLinksPattern.MatchString(link)
 }
 
+// FIXME: This function is not concurrent safe
 func getIssueFullPattern() *regexp.Regexp {
 	if issueFullPattern == nil {
 		issueFullPattern = regexp.MustCompile(regexp.QuoteMeta(setting.AppURL) +
@@ -333,40 +334,37 @@ func (ctx *postProcessCtx) postProcess(rawHTML []byte) ([]byte, error) {
 	_, _ = res.WriteString("</body></html>")
 
 	// parse the HTML
-	nodes, err := html.ParseFragment(res, nil)
+	node, err := html.Parse(res)
 	if err != nil {
 		return nil, &postProcessError{"invalid HTML", err}
 	}
 
-	for _, node := range nodes {
-		ctx.visitNode(node, true)
+	if node.Type == html.DocumentNode {
+		node = node.FirstChild
 	}
 
-	newNodes := make([]*html.Node, 0, len(nodes))
+	ctx.visitNode(node, true)
 
-	for _, node := range nodes {
-		if node.Data == "html" {
-			node = node.FirstChild
-			for node != nil && node.Data != "body" {
-				node = node.NextSibling
-			}
-		}
-		if node == nil {
-			continue
+	nodes := make([]*html.Node, 0, 5)
+
+	if node.Data == "html" {
+		node = node.FirstChild
+		for node != nil && node.Data != "body" {
+			node = node.NextSibling
 		}
+	}
+	if node != nil {
 		if node.Data == "body" {
 			child := node.FirstChild
 			for child != nil {
-				newNodes = append(newNodes, child)
+				nodes = append(nodes, child)
 				child = child.NextSibling
 			}
 		} else {
-			newNodes = append(newNodes, node)
+			nodes = append(nodes, node)
 		}
 	}
 
-	nodes = newNodes
-
 	// Create buffer in which the data will be placed again. We know that the
 	// length will be at least that of res; to spare a few alloc+copy, we
 	// reuse res, resetting its length to 0.
@@ -403,24 +401,20 @@ func (ctx *postProcessCtx) visitNode(node *html.Node, visitText bool) {
 		}
 	case html.ElementNode:
 		if node.Data == "img" {
-			attrs := node.Attr
-			for idx, attr := range attrs {
+			for i, attr := range node.Attr {
 				if attr.Key != "src" {
 					continue
 				}
-				link := []byte(attr.Val)
-				if len(link) > 0 && !IsLink(link) {
+				if len(attr.Val) > 0 && !isLinkStr(attr.Val) && !strings.HasPrefix(attr.Val, "data:image/") {
 					prefix := ctx.urlPrefix
 					if ctx.isWikiMarkdown {
 						prefix = util.URLJoin(prefix, "wiki", "raw")
 					}
 					prefix = strings.Replace(prefix, "/src/", "/media/", 1)
 
-					lnk := string(link)
-					lnk = util.URLJoin(prefix, lnk)
-					link = []byte(lnk)
+					attr.Val = util.URLJoin(prefix, attr.Val)
 				}
-				node.Attr[idx].Val = string(link)
+				node.Attr[i] = attr
 			}
 		} else if node.Data == "a" {
 			visitText = false
@@ -610,26 +604,38 @@ func replaceContentList(node *html.Node, i, j int, newNodes []*html.Node) {
 }
 
 func mentionProcessor(ctx *postProcessCtx, node *html.Node) {
-	// We replace only the first mention; other mentions will be addressed later
-	found, loc := references.FindFirstMentionBytes([]byte(node.Data))
-	if !found {
-		return
-	}
-	mention := node.Data[loc.Start:loc.End]
-	var teams string
-	teams, ok := ctx.metas["teams"]
-	// FIXME: util.URLJoin may not be necessary here:
-	// - setting.AppURL is defined to have a terminal '/' so unless mention[1:]
-	// is an AppSubURL link we can probably fallback to concatenation.
-	// team mention should follow @orgName/teamName style
-	if ok && strings.Contains(mention, "/") {
-		mentionOrgAndTeam := strings.Split(mention, "/")
-		if mentionOrgAndTeam[0][1:] == ctx.metas["org"] && strings.Contains(teams, ","+strings.ToLower(mentionOrgAndTeam[1])+",") {
-			replaceContent(node, loc.Start, loc.End, createLink(util.URLJoin(setting.AppURL, "org", ctx.metas["org"], "teams", mentionOrgAndTeam[1]), mention, "mention"))
+	start := 0
+	next := node.NextSibling
+	for node != nil && node != next && start < len(node.Data) {
+		// We replace only the first mention; other mentions will be addressed later
+		found, loc := references.FindFirstMentionBytes([]byte(node.Data[start:]))
+		if !found {
+			return
 		}
-		return
+		loc.Start += start
+		loc.End += start
+		mention := node.Data[loc.Start:loc.End]
+		var teams string
+		teams, ok := ctx.metas["teams"]
+		// FIXME: util.URLJoin may not be necessary here:
+		// - setting.AppURL is defined to have a terminal '/' so unless mention[1:]
+		// is an AppSubURL link we can probably fallback to concatenation.
+		// team mention should follow @orgName/teamName style
+		if ok && strings.Contains(mention, "/") {
+			mentionOrgAndTeam := strings.Split(mention, "/")
+			if mentionOrgAndTeam[0][1:] == ctx.metas["org"] && strings.Contains(teams, ","+strings.ToLower(mentionOrgAndTeam[1])+",") {
+				replaceContent(node, loc.Start, loc.End, createLink(util.URLJoin(setting.AppURL, "org", ctx.metas["org"], "teams", mentionOrgAndTeam[1]), mention, "mention"))
+				node = node.NextSibling.NextSibling
+				start = 0
+				continue
+			}
+			start = loc.End
+			continue
+		}
+		replaceContent(node, loc.Start, loc.End, createLink(util.URLJoin(setting.AppURL, mention[1:]), mention, "mention"))
+		node = node.NextSibling.NextSibling
+		start = 0
 	}
-	replaceContent(node, loc.Start, loc.End, createLink(util.URLJoin(setting.AppURL, mention[1:]), mention, "mention"))
 }
 
 func shortLinkProcessor(ctx *postProcessCtx, node *html.Node) {
@@ -637,188 +643,195 @@ func shortLinkProcessor(ctx *postProcessCtx, node *html.Node) {
 }
 
 func shortLinkProcessorFull(ctx *postProcessCtx, node *html.Node, noLink bool) {
-	m := shortLinkPattern.FindStringSubmatchIndex(node.Data)
-	if m == nil {
-		return
-	}
+	next := node.NextSibling
+	for node != nil && node != next {
+		m := shortLinkPattern.FindStringSubmatchIndex(node.Data)
+		if m == nil {
+			return
+		}
 
-	content := node.Data[m[2]:m[3]]
-	tail := node.Data[m[4]:m[5]]
-	props := make(map[string]string)
+		content := node.Data[m[2]:m[3]]
+		tail := node.Data[m[4]:m[5]]
+		props := make(map[string]string)
 
-	// MediaWiki uses [[link|text]], while GitHub uses [[text|link]]
-	// It makes page handling terrible, but we prefer GitHub syntax
-	// And fall back to MediaWiki only when it is obvious from the look
-	// Of text and link contents
-	sl := strings.Split(content, "|")
-	for _, v := range sl {
-		if equalPos := strings.IndexByte(v, '='); equalPos == -1 {
-			// There is no equal in this argument; this is a mandatory arg
-			if props["name"] == "" {
-				if isLinkStr(v) {
-					// If we clearly see it is a link, we save it so
+		// MediaWiki uses [[link|text]], while GitHub uses [[text|link]]
+		// It makes page handling terrible, but we prefer GitHub syntax
+		// And fall back to MediaWiki only when it is obvious from the look
+		// Of text and link contents
+		sl := strings.Split(content, "|")
+		for _, v := range sl {
+			if equalPos := strings.IndexByte(v, '='); equalPos == -1 {
+				// There is no equal in this argument; this is a mandatory arg
+				if props["name"] == "" {
+					if isLinkStr(v) {
+						// If we clearly see it is a link, we save it so
 
-					// But first we need to ensure, that if both mandatory args provided
-					// look like links, we stick to GitHub syntax
-					if props["link"] != "" {
-						props["name"] = props["link"]
+						// But first we need to ensure, that if both mandatory args provided
+						// look like links, we stick to GitHub syntax
+						if props["link"] != "" {
+							props["name"] = props["link"]
+						}
+
+						props["link"] = strings.TrimSpace(v)
+					} else {
+						props["name"] = v
 					}
-
-					props["link"] = strings.TrimSpace(v)
 				} else {
-					props["name"] = v
+					props["link"] = strings.TrimSpace(v)
 				}
 			} else {
-				props["link"] = strings.TrimSpace(v)
-			}
-		} else {
-			// There is an equal; optional argument.
+				// There is an equal; optional argument.
 
-			sep := strings.IndexByte(v, '=')
-			key, val := v[:sep], html.UnescapeString(v[sep+1:])
+				sep := strings.IndexByte(v, '=')
+				key, val := v[:sep], html.UnescapeString(v[sep+1:])
 
-			// When parsing HTML, x/net/html will change all quotes which are
-			// not used for syntax into UTF-8 quotes. So checking val[0] won't
-			// be enough, since that only checks a single byte.
-			if len(val) > 1 {
-				if (strings.HasPrefix(val, "“") && strings.HasSuffix(val, "”")) ||
-					(strings.HasPrefix(val, "‘") && strings.HasSuffix(val, "’")) {
-					const lenQuote = len("‘")
-					val = val[lenQuote : len(val)-lenQuote]
-				} else if (strings.HasPrefix(val, "\"") && strings.HasSuffix(val, "\"")) ||
-					(strings.HasPrefix(val, "'") && strings.HasSuffix(val, "'")) {
-					val = val[1 : len(val)-1]
-				} else if strings.HasPrefix(val, "'") && strings.HasSuffix(val, "’") {
-					const lenQuote = len("‘")
-					val = val[1 : len(val)-lenQuote]
+				// When parsing HTML, x/net/html will change all quotes which are
+				// not used for syntax into UTF-8 quotes. So checking val[0] won't
+				// be enough, since that only checks a single byte.
+				if len(val) > 1 {
+					if (strings.HasPrefix(val, "“") && strings.HasSuffix(val, "”")) ||
+						(strings.HasPrefix(val, "‘") && strings.HasSuffix(val, "’")) {
+						const lenQuote = len("‘")
+						val = val[lenQuote : len(val)-lenQuote]
+					} else if (strings.HasPrefix(val, "\"") && strings.HasSuffix(val, "\"")) ||
+						(strings.HasPrefix(val, "'") && strings.HasSuffix(val, "'")) {
+						val = val[1 : len(val)-1]
+					} else if strings.HasPrefix(val, "'") && strings.HasSuffix(val, "’") {
+						const lenQuote = len("‘")
+						val = val[1 : len(val)-lenQuote]
+					}
 				}
+				props[key] = val
 			}
-			props[key] = val
 		}
-	}
 
-	var name, link string
-	if props["link"] != "" {
-		link = props["link"]
-	} else if props["name"] != "" {
-		link = props["name"]
-	}
-	if props["title"] != "" {
-		name = props["title"]
-	} else if props["name"] != "" {
-		name = props["name"]
-	} else {
-		name = link
-	}
-
-	name += tail
-	image := false
-	switch ext := filepath.Ext(link); ext {
-	// fast path: empty string, ignore
-	case "":
-		break
-	case ".jpg", ".jpeg", ".png", ".tif", ".tiff", ".webp", ".gif", ".bmp", ".ico", ".svg":
-		image = true
-	}
-
-	childNode := &html.Node{}
-	linkNode := &html.Node{
-		FirstChild: childNode,
-		LastChild:  childNode,
-		Type:       html.ElementNode,
-		Data:       "a",
-		DataAtom:   atom.A,
-	}
-	childNode.Parent = linkNode
-	absoluteLink := isLinkStr(link)
-	if !absoluteLink {
-		if image {
-			link = strings.ReplaceAll(link, " ", "+")
+		var name, link string
+		if props["link"] != "" {
+			link = props["link"]
+		} else if props["name"] != "" {
+			link = props["name"]
+		}
+		if props["title"] != "" {
+			name = props["title"]
+		} else if props["name"] != "" {
+			name = props["name"]
 		} else {
-			link = strings.ReplaceAll(link, " ", "-")
-		}
-		if !strings.Contains(link, "/") {
-			link = url.PathEscape(link)
-		}
-	}
-	urlPrefix := ctx.urlPrefix
-	if image {
-		if !absoluteLink {
-			if IsSameDomain(urlPrefix) {
-				urlPrefix = strings.Replace(urlPrefix, "/src/", "/raw/", 1)
-			}
-			if ctx.isWikiMarkdown {
-				link = util.URLJoin("wiki", "raw", link)
-			}
-			link = util.URLJoin(urlPrefix, link)
-		}
-		title := props["title"]
-		if title == "" {
-			title = props["alt"]
-		}
-		if title == "" {
-			title = path.Base(name)
-		}
-		alt := props["alt"]
-		if alt == "" {
-			alt = name
+			name = link
 		}
 
-		// make the childNode an image - if we can, we also place the alt
-		childNode.Type = html.ElementNode
-		childNode.Data = "img"
-		childNode.DataAtom = atom.Img
-		childNode.Attr = []html.Attribute{
-			{Key: "src", Val: link},
-			{Key: "title", Val: title},
-			{Key: "alt", Val: alt},
+		name += tail
+		image := false
+		switch ext := filepath.Ext(link); ext {
+		// fast path: empty string, ignore
+		case "":
+			// leave image as false
+		case ".jpg", ".jpeg", ".png", ".tif", ".tiff", ".webp", ".gif", ".bmp", ".ico", ".svg":
+			image = true
 		}
-		if alt == "" {
-			childNode.Attr = childNode.Attr[:2]
+
+		childNode := &html.Node{}
+		linkNode := &html.Node{
+			FirstChild: childNode,
+			LastChild:  childNode,
+			Type:       html.ElementNode,
+			Data:       "a",
+			DataAtom:   atom.A,
 		}
-	} else {
+		childNode.Parent = linkNode
+		absoluteLink := isLinkStr(link)
 		if !absoluteLink {
-			if ctx.isWikiMarkdown {
-				link = util.URLJoin("wiki", link)
+			if image {
+				link = strings.ReplaceAll(link, " ", "+")
+			} else {
+				link = strings.ReplaceAll(link, " ", "-")
+			}
+			if !strings.Contains(link, "/") {
+				link = url.PathEscape(link)
 			}
-			link = util.URLJoin(urlPrefix, link)
 		}
-		childNode.Type = html.TextNode
-		childNode.Data = name
+		urlPrefix := ctx.urlPrefix
+		if image {
+			if !absoluteLink {
+				if IsSameDomain(urlPrefix) {
+					urlPrefix = strings.Replace(urlPrefix, "/src/", "/raw/", 1)
+				}
+				if ctx.isWikiMarkdown {
+					link = util.URLJoin("wiki", "raw", link)
+				}
+				link = util.URLJoin(urlPrefix, link)
+			}
+			title := props["title"]
+			if title == "" {
+				title = props["alt"]
+			}
+			if title == "" {
+				title = path.Base(name)
+			}
+			alt := props["alt"]
+			if alt == "" {
+				alt = name
+			}
+
+			// make the childNode an image - if we can, we also place the alt
+			childNode.Type = html.ElementNode
+			childNode.Data = "img"
+			childNode.DataAtom = atom.Img
+			childNode.Attr = []html.Attribute{
+				{Key: "src", Val: link},
+				{Key: "title", Val: title},
+				{Key: "alt", Val: alt},
+			}
+			if alt == "" {
+				childNode.Attr = childNode.Attr[:2]
+			}
+		} else {
+			if !absoluteLink {
+				if ctx.isWikiMarkdown {
+					link = util.URLJoin("wiki", link)
+				}
+				link = util.URLJoin(urlPrefix, link)
+			}
+			childNode.Type = html.TextNode
+			childNode.Data = name
+		}
+		if noLink {
+			linkNode = childNode
+		} else {
+			linkNode.Attr = []html.Attribute{{Key: "href", Val: link}}
+		}
+		replaceContent(node, m[0], m[1], linkNode)
+		node = node.NextSibling.NextSibling
 	}
-	if noLink {
-		linkNode = childNode
-	} else {
-		linkNode.Attr = []html.Attribute{{Key: "href", Val: link}}
-	}
-	replaceContent(node, m[0], m[1], linkNode)
 }
 
 func fullIssuePatternProcessor(ctx *postProcessCtx, node *html.Node) {
 	if ctx.metas == nil {
 		return
 	}
-	m := getIssueFullPattern().FindStringSubmatchIndex(node.Data)
-	if m == nil {
-		return
-	}
-	link := node.Data[m[0]:m[1]]
-	id := "#" + node.Data[m[2]:m[3]]
+	next := node.NextSibling
+	for node != nil && node != next {
+		m := getIssueFullPattern().FindStringSubmatchIndex(node.Data)
+		if m == nil {
+			return
+		}
+		link := node.Data[m[0]:m[1]]
+		id := "#" + node.Data[m[2]:m[3]]
 
-	// extract repo and org name from matched link like
-	// http://localhost:3000/gituser/myrepo/issues/1
-	linkParts := strings.Split(path.Clean(link), "/")
-	matchOrg := linkParts[len(linkParts)-4]
-	matchRepo := linkParts[len(linkParts)-3]
+		// extract repo and org name from matched link like
+		// http://localhost:3000/gituser/myrepo/issues/1
+		linkParts := strings.Split(path.Clean(link), "/")
+		matchOrg := linkParts[len(linkParts)-4]
+		matchRepo := linkParts[len(linkParts)-3]
 
-	if matchOrg == ctx.metas["user"] && matchRepo == ctx.metas["repo"] {
-		// TODO if m[4]:m[5] is not nil, then link is to a comment,
-		// and we should indicate that in the text somehow
-		replaceContent(node, m[0], m[1], createLink(link, id, "ref-issue"))
-
-	} else {
-		orgRepoID := matchOrg + "/" + matchRepo + id
-		replaceContent(node, m[0], m[1], createLink(link, orgRepoID, "ref-issue"))
+		if matchOrg == ctx.metas["user"] && matchRepo == ctx.metas["repo"] {
+			// TODO if m[4]:m[5] is not nil, then link is to a comment,
+			// and we should indicate that in the text somehow
+			replaceContent(node, m[0], m[1], createLink(link, id, "ref-issue"))
+		} else {
+			orgRepoID := matchOrg + "/" + matchRepo + id
+			replaceContent(node, m[0], m[1], createLink(link, orgRepoID, "ref-issue"))
+		}
+		node = node.NextSibling.NextSibling
 	}
 }
 
@@ -826,70 +839,74 @@ func issueIndexPatternProcessor(ctx *postProcessCtx, node *html.Node) {
 	if ctx.metas == nil {
 		return
 	}
-
 	var (
 		found bool
 		ref   *references.RenderizableReference
 	)
 
-	_, exttrack := ctx.metas["format"]
-	alphanum := ctx.metas["style"] == IssueNameStyleAlphanumeric
+	next := node.NextSibling
+	for node != nil && node != next {
+		_, exttrack := ctx.metas["format"]
+		alphanum := ctx.metas["style"] == IssueNameStyleAlphanumeric
 
-	// Repos with external issue trackers might still need to reference local PRs
-	// We need to concern with the first one that shows up in the text, whichever it is
-	found, ref = references.FindRenderizableReferenceNumeric(node.Data, exttrack && alphanum)
-	if exttrack && alphanum {
-		if found2, ref2 := references.FindRenderizableReferenceAlphanumeric(node.Data); found2 {
-			if !found || ref2.RefLocation.Start < ref.RefLocation.Start {
-				found = true
-				ref = ref2
+		// Repos with external issue trackers might still need to reference local PRs
+		// We need to concern with the first one that shows up in the text, whichever it is
+		found, ref = references.FindRenderizableReferenceNumeric(node.Data, exttrack && alphanum)
+		if exttrack && alphanum {
+			if found2, ref2 := references.FindRenderizableReferenceAlphanumeric(node.Data); found2 {
+				if !found || ref2.RefLocation.Start < ref.RefLocation.Start {
+					found = true
+					ref = ref2
+				}
 			}
 		}
-	}
-	if !found {
-		return
-	}
-
-	var link *html.Node
-	reftext := node.Data[ref.RefLocation.Start:ref.RefLocation.End]
-	if exttrack && !ref.IsPull {
-		ctx.metas["index"] = ref.Issue
-		link = createLink(com.Expand(ctx.metas["format"], ctx.metas), reftext, "ref-issue")
-	} else {
-		// Path determines the type of link that will be rendered. It's unknown at this point whether
-		// the linked item is actually a PR or an issue. Luckily it's of no real consequence because
-		// Gitea will redirect on click as appropriate.
-		path := "issues"
-		if ref.IsPull {
-			path = "pulls"
+		if !found {
+			return
 		}
-		if ref.Owner == "" {
-			link = createLink(util.URLJoin(setting.AppURL, ctx.metas["user"], ctx.metas["repo"], path, ref.Issue), reftext, "ref-issue")
+
+		var link *html.Node
+		reftext := node.Data[ref.RefLocation.Start:ref.RefLocation.End]
+		if exttrack && !ref.IsPull {
+			ctx.metas["index"] = ref.Issue
+			link = createLink(com.Expand(ctx.metas["format"], ctx.metas), reftext, "ref-issue")
 		} else {
-			link = createLink(util.URLJoin(setting.AppURL, ref.Owner, ref.Name, path, ref.Issue), reftext, "ref-issue")
+			// Path determines the type of link that will be rendered. It's unknown at this point whether
+			// the linked item is actually a PR or an issue. Luckily it's of no real consequence because
+			// Gitea will redirect on click as appropriate.
+			path := "issues"
+			if ref.IsPull {
+				path = "pulls"
+			}
+			if ref.Owner == "" {
+				link = createLink(util.URLJoin(setting.AppURL, ctx.metas["user"], ctx.metas["repo"], path, ref.Issue), reftext, "ref-issue")
+			} else {
+				link = createLink(util.URLJoin(setting.AppURL, ref.Owner, ref.Name, path, ref.Issue), reftext, "ref-issue")
+			}
 		}
-	}
 
-	if ref.Action == references.XRefActionNone {
-		replaceContent(node, ref.RefLocation.Start, ref.RefLocation.End, link)
-		return
-	}
+		if ref.Action == references.XRefActionNone {
+			replaceContent(node, ref.RefLocation.Start, ref.RefLocation.End, link)
+			node = node.NextSibling.NextSibling
+			continue
+		}
 
-	// Decorate action keywords if actionable
-	var keyword *html.Node
-	if references.IsXrefActionable(ref, exttrack, alphanum) {
-		keyword = createKeyword(node.Data[ref.ActionLocation.Start:ref.ActionLocation.End])
-	} else {
-		keyword = &html.Node{
+		// Decorate action keywords if actionable
+		var keyword *html.Node
+		if references.IsXrefActionable(ref, exttrack, alphanum) {
+			keyword = createKeyword(node.Data[ref.ActionLocation.Start:ref.ActionLocation.End])
+		} else {
+			keyword = &html.Node{
+				Type: html.TextNode,
+				Data: node.Data[ref.ActionLocation.Start:ref.ActionLocation.End],
+			}
+		}
+		spaces := &html.Node{
 			Type: html.TextNode,
-			Data: node.Data[ref.ActionLocation.Start:ref.ActionLocation.End],
+			Data: node.Data[ref.ActionLocation.End:ref.RefLocation.Start],
 		}
+		replaceContentList(node, ref.ActionLocation.Start, ref.RefLocation.End, []*html.Node{keyword, spaces, link})
+		node = node.NextSibling.NextSibling.NextSibling.NextSibling
 	}
-	spaces := &html.Node{
-		Type: html.TextNode,
-		Data: node.Data[ref.ActionLocation.End:ref.RefLocation.Start],
-	}
-	replaceContentList(node, ref.ActionLocation.Start, ref.RefLocation.End, []*html.Node{keyword, spaces, link})
 }
 
 // fullSha1PatternProcessor renders SHA containing URLs
@@ -897,87 +914,112 @@ func fullSha1PatternProcessor(ctx *postProcessCtx, node *html.Node) {
 	if ctx.metas == nil {
 		return
 	}
-	m := anySHA1Pattern.FindStringSubmatchIndex(node.Data)
-	if m == nil {
-		return
-	}
 
-	urlFull := node.Data[m[0]:m[1]]
-	text := base.ShortSha(node.Data[m[2]:m[3]])
-
-	// 3rd capture group matches a optional path
-	subpath := ""
-	if m[5] > 0 {
-		subpath = node.Data[m[4]:m[5]]
-	}
-
-	// 4th capture group matches a optional url hash
-	hash := ""
-	if m[7] > 0 {
-		hash = node.Data[m[6]:m[7]][1:]
-	}
-
-	start := m[0]
-	end := m[1]
-
-	// If url ends in '.', it's very likely that it is not part of the
-	// actual url but used to finish a sentence.
-	if strings.HasSuffix(urlFull, ".") {
-		end--
-		urlFull = urlFull[:len(urlFull)-1]
-		if hash != "" {
-			hash = hash[:len(hash)-1]
-		} else if subpath != "" {
-			subpath = subpath[:len(subpath)-1]
+	next := node.NextSibling
+	for node != nil && node != next {
+		m := anySHA1Pattern.FindStringSubmatchIndex(node.Data)
+		if m == nil {
+			return
 		}
-	}
 
-	if subpath != "" {
-		text += subpath
-	}
+		urlFull := node.Data[m[0]:m[1]]
+		text := base.ShortSha(node.Data[m[2]:m[3]])
 
-	if hash != "" {
-		text += " (" + hash + ")"
-	}
+		// 3rd capture group matches a optional path
+		subpath := ""
+		if m[5] > 0 {
+			subpath = node.Data[m[4]:m[5]]
+		}
 
-	replaceContent(node, start, end, createCodeLink(urlFull, text, "commit"))
+		// 4th capture group matches a optional url hash
+		hash := ""
+		if m[7] > 0 {
+			hash = node.Data[m[6]:m[7]][1:]
+		}
+
+		start := m[0]
+		end := m[1]
+
+		// If url ends in '.', it's very likely that it is not part of the
+		// actual url but used to finish a sentence.
+		if strings.HasSuffix(urlFull, ".") {
+			end--
+			urlFull = urlFull[:len(urlFull)-1]
+			if hash != "" {
+				hash = hash[:len(hash)-1]
+			} else if subpath != "" {
+				subpath = subpath[:len(subpath)-1]
+			}
+		}
+
+		if subpath != "" {
+			text += subpath
+		}
+
+		if hash != "" {
+			text += " (" + hash + ")"
+		}
+
+		replaceContent(node, start, end, createCodeLink(urlFull, text, "commit"))
+		node = node.NextSibling.NextSibling
+	}
 }
 
 // emojiShortCodeProcessor for rendering text like :smile: into emoji
 func emojiShortCodeProcessor(ctx *postProcessCtx, node *html.Node) {
-
-	m := EmojiShortCodeRegex.FindStringSubmatchIndex(node.Data)
-	if m == nil {
-		return
-	}
-
-	alias := node.Data[m[0]:m[1]]
-	alias = strings.ReplaceAll(alias, ":", "")
-	converted := emoji.FromAlias(alias)
-	if converted == nil {
-		// check if this is a custom reaction
-		s := strings.Join(setting.UI.Reactions, " ") + "gitea"
-		if strings.Contains(s, alias) {
-			replaceContent(node, m[0], m[1], createCustomEmoji(alias, "emoji"))
+	start := 0
+	next := node.NextSibling
+	for node != nil && node != next && start < len(node.Data) {
+		m := EmojiShortCodeRegex.FindStringSubmatchIndex(node.Data[start:])
+		if m == nil {
 			return
 		}
-		return
-	}
+		m[0] += start
+		m[1] += start
 
-	replaceContent(node, m[0], m[1], createEmoji(converted.Emoji, "emoji", converted.Description))
+		start = m[1]
+
+		alias := node.Data[m[0]:m[1]]
+		alias = strings.ReplaceAll(alias, ":", "")
+		converted := emoji.FromAlias(alias)
+		if converted == nil {
+			// check if this is a custom reaction
+			s := strings.Join(setting.UI.Reactions, " ") + "gitea"
+			if strings.Contains(s, alias) {
+				replaceContent(node, m[0], m[1], createCustomEmoji(alias, "emoji"))
+				node = node.NextSibling.NextSibling
+				start = 0
+				continue
+			}
+			continue
+		}
+
+		replaceContent(node, m[0], m[1], createEmoji(converted.Emoji, "emoji", converted.Description))
+		node = node.NextSibling.NextSibling
+		start = 0
+	}
 }
 
 // emoji processor to match emoji and add emoji class
 func emojiProcessor(ctx *postProcessCtx, node *html.Node) {
-	m := emoji.FindEmojiSubmatchIndex(node.Data)
-	if m == nil {
-		return
-	}
+	start := 0
+	next := node.NextSibling
+	for node != nil && node != next && start < len(node.Data) {
+		m := emoji.FindEmojiSubmatchIndex(node.Data[start:])
+		if m == nil {
+			return
+		}
+		m[0] += start
+		m[1] += start
 
-	codepoint := node.Data[m[0]:m[1]]
-	val := emoji.FromCode(codepoint)
-	if val != nil {
-		replaceContent(node, m[0], m[1], createEmoji(codepoint, "emoji", val.Description))
+		codepoint := node.Data[m[0]:m[1]]
+		start = m[1]
+		val := emoji.FromCode(codepoint)
+		if val != nil {
+			replaceContent(node, m[0], m[1], createEmoji(codepoint, "emoji", val.Description))
+			node = node.NextSibling.NextSibling
+			start = 0
+		}
 	}
 }
 
@@ -987,49 +1029,69 @@ func sha1CurrentPatternProcessor(ctx *postProcessCtx, node *html.Node) {
 	if ctx.metas == nil || ctx.metas["user"] == "" || ctx.metas["repo"] == "" || ctx.metas["repoPath"] == "" {
 		return
 	}
-	m := sha1CurrentPattern.FindStringSubmatchIndex(node.Data)
-	if m == nil {
-		return
-	}
-	hash := node.Data[m[2]:m[3]]
-	// The regex does not lie, it matches the hash pattern.
-	// However, a regex cannot know if a hash actually exists or not.
-	// We could assume that a SHA1 hash should probably contain alphas AND numerics
-	// but that is not always the case.
-	// Although unlikely, deadbeef and 1234567 are valid short forms of SHA1 hash
-	// as used by git and github for linking and thus we have to do similar.
-	// Because of this, we check to make sure that a matched hash is actually
-	// a commit in the repository before making it a link.
-	if _, err := git.NewCommand("rev-parse", "--verify", hash).RunInDirBytes(ctx.metas["repoPath"]); err != nil {
-		if !strings.Contains(err.Error(), "fatal: Needed a single revision") {
-			log.Debug("sha1CurrentPatternProcessor git rev-parse: %v", err)
-		}
-		return
-	}
 
-	replaceContent(node, m[2], m[3],
-		createCodeLink(util.URLJoin(setting.AppURL, ctx.metas["user"], ctx.metas["repo"], "commit", hash), base.ShortSha(hash), "commit"))
+	start := 0
+	next := node.NextSibling
+	for node != nil && node != next && start < len(node.Data) {
+		m := sha1CurrentPattern.FindStringSubmatchIndex(node.Data[start:])
+		if m == nil {
+			return
+		}
+		m[2] += start
+		m[3] += start
+
+		hash := node.Data[m[2]:m[3]]
+		// The regex does not lie, it matches the hash pattern.
+		// However, a regex cannot know if a hash actually exists or not.
+		// We could assume that a SHA1 hash should probably contain alphas AND numerics
+		// but that is not always the case.
+		// Although unlikely, deadbeef and 1234567 are valid short forms of SHA1 hash
+		// as used by git and github for linking and thus we have to do similar.
+		// Because of this, we check to make sure that a matched hash is actually
+		// a commit in the repository before making it a link.
+		if _, err := git.NewCommand("rev-parse", "--verify", hash).RunInDirBytes(ctx.metas["repoPath"]); err != nil {
+			if !strings.Contains(err.Error(), "fatal: Needed a single revision") {
+				log.Debug("sha1CurrentPatternProcessor git rev-parse: %v", err)
+			}
+			start = m[3]
+			continue
+		}
+
+		replaceContent(node, m[2], m[3],
+			createCodeLink(util.URLJoin(setting.AppURL, ctx.metas["user"], ctx.metas["repo"], "commit", hash), base.ShortSha(hash), "commit"))
+		start = 0
+		node = node.NextSibling.NextSibling
+	}
 }
 
 // emailAddressProcessor replaces raw email addresses with a mailto: link.
 func emailAddressProcessor(ctx *postProcessCtx, node *html.Node) {
-	m := emailRegex.FindStringSubmatchIndex(node.Data)
-	if m == nil {
-		return
+	next := node.NextSibling
+	for node != nil && node != next {
+		m := emailRegex.FindStringSubmatchIndex(node.Data)
+		if m == nil {
+			return
+		}
+
+		mail := node.Data[m[2]:m[3]]
+		replaceContent(node, m[2], m[3], createLink("mailto:"+mail, mail, "mailto"))
+		node = node.NextSibling.NextSibling
 	}
-	mail := node.Data[m[2]:m[3]]
-	replaceContent(node, m[2], m[3], createLink("mailto:"+mail, mail, "mailto"))
 }
 
 // linkProcessor creates links for any HTTP or HTTPS URL not captured by
 // markdown.
 func linkProcessor(ctx *postProcessCtx, node *html.Node) {
-	m := common.LinkRegex.FindStringIndex(node.Data)
-	if m == nil {
-		return
+	next := node.NextSibling
+	for node != nil && node != next {
+		m := common.LinkRegex.FindStringIndex(node.Data)
+		if m == nil {
+			return
+		}
+		uri := node.Data[m[0]:m[1]]
+		replaceContent(node, m[0], m[1], createLink(uri, uri, "link"))
+		node = node.NextSibling.NextSibling
 	}
-	uri := node.Data[m[0]:m[1]]
-	replaceContent(node, m[0], m[1], createLink(uri, uri, "link"))
 }
 
 func genDefaultLinkProcessor(defaultLink string) processor {
@@ -1053,12 +1115,17 @@ func genDefaultLinkProcessor(defaultLink string) processor {
 
 // descriptionLinkProcessor creates links for DescriptionHTML
 func descriptionLinkProcessor(ctx *postProcessCtx, node *html.Node) {
-	m := common.LinkRegex.FindStringIndex(node.Data)
-	if m == nil {
-		return
+	next := node.NextSibling
+	for node != nil && node != next {
+		m := common.LinkRegex.FindStringIndex(node.Data)
+		if m == nil {
+			return
+		}
+
+		uri := node.Data[m[0]:m[1]]
+		replaceContent(node, m[0], m[1], createDescriptionLink(uri, uri))
+		node = node.NextSibling.NextSibling
 	}
-	uri := node.Data[m[0]:m[1]]
-	replaceContent(node, m[0], m[1], createDescriptionLink(uri, uri))
 }
 
 func createDescriptionLink(href, content string) *html.Node {

modules/markup/html_test.go

@@ -124,13 +124,13 @@ func TestRender_links(t *testing.T) {
 		`<p><a href="http://www.example.com/wpstyle/?p=364" rel="nofollow">http://www.example.com/wpstyle/?p=364</a></p>`)
 	test(
 		"https://www.example.com/foo/?bar=baz&inga=42&quux",
-		`<p><a href="https://www.example.com/foo/?bar=baz&inga=42&quux" rel="nofollow">https://www.example.com/foo/?bar=baz&amp;inga=42&amp;quux</a></p>`)
+		`<p><a href="https://www.example.com/foo/?bar=baz&amp;inga=42&amp;quux" rel="nofollow">https://www.example.com/foo/?bar=baz&amp;inga=42&amp;quux</a></p>`)
 	test(
 		"http://142.42.1.1/",
 		`<p><a href="http://142.42.1.1/" rel="nofollow">http://142.42.1.1/</a></p>`)
 	test(
 		"https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd",
-		`<p><a href="https://github.com/go-gitea/gitea/?p=aaa%2Fbbb.html#ccc-ddd" rel="nofollow">https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd</a></p>`)
+		`<p><a href="https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd" rel="nofollow">https://github.com/go-gitea/gitea/?p=aaa/bbb.html#ccc-ddd</a></p>`)
 	test(
 		"https://en.wikipedia.org/wiki/URL_(disambiguation)",
 		`<p><a href="https://en.wikipedia.org/wiki/URL_(disambiguation)" rel="nofollow">https://en.wikipedia.org/wiki/URL_(disambiguation)</a></p>`)
@@ -148,7 +148,7 @@ func TestRender_links(t *testing.T) {
 		`<p><a href="ftp://gitea.com/file.txt" rel="nofollow">ftp://gitea.com/file.txt</a></p>`)
 	test(
 		"magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&dn=download",
-		`<p><a href="magnet:?xt=urn%3Abtih%3A5dee65101db281ac9c46344cd6b175cdcadabcde&dn=download" rel="nofollow">magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&amp;dn=download</a></p>`)
+		`<p><a href="magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&amp;dn=download" rel="nofollow">magnet:?xt=urn:btih:5dee65101db281ac9c46344cd6b175cdcadabcde&amp;dn=download</a></p>`)
 
 	// Test that should *not* be turned into URL
 	test(
@@ -384,6 +384,32 @@ func TestRender_ShortLinks(t *testing.T) {
 		`<p><a href="https://example.org" rel="nofollow">[[foobar]]</a></p>`)
 }
 
+func TestRender_RelativeImages(t *testing.T) {
+	setting.AppURL = AppURL
+	setting.AppSubURL = AppSubURL
+	tree := util.URLJoin(AppSubURL, "src", "master")
+
+	test := func(input, expected, expectedWiki string) {
+		buffer := markdown.RenderString(input, tree, localMetas)
+		assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
+		buffer = markdown.RenderWiki([]byte(input), setting.AppSubURL, localMetas)
+		assert.Equal(t, strings.TrimSpace(expectedWiki), strings.TrimSpace(buffer))
+	}
+
+	rawwiki := util.URLJoin(AppSubURL, "wiki", "raw")
+	mediatree := util.URLJoin(AppSubURL, "media", "master")
+
+	test(
+		`<img src="Link">`,
+		`<img src="`+util.URLJoin(mediatree, "Link")+`"/>`,
+		`<img src="`+util.URLJoin(rawwiki, "Link")+`"/>`)
+
+	test(
+		`<img src="./icon.png">`,
+		`<img src="`+util.URLJoin(mediatree, "icon.png")+`"/>`,
+		`<img src="`+util.URLJoin(rawwiki, "icon.png")+`"/>`)
+}
+
 func Test_ParseClusterFuzz(t *testing.T) {
 	setting.AppURL = AppURL
 	setting.AppSubURL = AppSubURL
@@ -408,3 +434,36 @@ func Test_ParseClusterFuzz(t *testing.T) {
 
 	assert.NotContains(t, string(val), "<html")
 }
+
+func TestIssue16020(t *testing.T) {
+	setting.AppURL = AppURL
+	setting.AppSubURL = AppSubURL
+
+	var localMetas = map[string]string{
+		"user": "go-gitea",
+		"repo": "gitea",
+	}
+
+	data := `<img src="data:image/png;base64,i//V"/>`
+
+	// func PostProcess(rawHTML []byte, urlPrefix string, metas map[string]string, isWikiMarkdown bool) ([]byte, error)
+	res, err := PostProcess([]byte(data), "https://example.com", localMetas, false)
+	assert.NoError(t, err)
+	assert.Equal(t, data, string(res))
+}
+
+func BenchmarkEmojiPostprocess(b *testing.B) {
+	data := "🥰 "
+	for len(data) < 1<<16 {
+		data += data
+	}
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		_, err := PostProcess(
+			[]byte(data),
+			"https://example.com",
+			localMetas,
+			false)
+		assert.NoError(b, err)
+	}
+}

modules/markup/sanitizer.go

@@ -50,9 +50,6 @@ func ReplaceSanitizer() {
 		sanitizer.policy.AllowURLSchemes(setting.Markdown.CustomURLSchemes...)
 	}
 
-	// Allow keyword markup
-	sanitizer.policy.AllowAttrs("class").Matching(regexp.MustCompile(`^` + keywordClass + `$`)).OnElements("span")
-
 	// Allow classes for anchors
 	sanitizer.policy.AllowAttrs("class").Matching(regexp.MustCompile(`ref-issue`)).OnElements("a")
 
@@ -68,8 +65,8 @@ func ReplaceSanitizer() {
 	// Allow classes for emojis
 	sanitizer.policy.AllowAttrs("class").Matching(regexp.MustCompile(`emoji`)).OnElements("img")
 
-	// Allow icons, emojis, and chroma syntax on span
-	sanitizer.policy.AllowAttrs("class").Matching(regexp.MustCompile(`^((icon(\s+[\p{L}\p{N}_-]+)+)|(emoji))$|^([a-z][a-z0-9]{0,2})$`)).OnElements("span")
+	// Allow icons, emojis, chroma syntax and keyword markup on span
+	sanitizer.policy.AllowAttrs("class").Matching(regexp.MustCompile(`^((icon(\s+[\p{L}\p{N}_-]+)+)|(emoji))$|^([a-z][a-z0-9]{0,2})$|^` + keywordClass + `$`)).OnElements("span")
 
 	// Allow generally safe attributes
 	generalSafeAttrs := []string{"abbr", "accept", "accept-charset",

modules/migrations/base/options.go

@@ -11,10 +11,13 @@ import "code.gitea.io/gitea/modules/structs"
 // this is for internal usage by migrations module and func who interact with it
 type MigrateOptions struct {
 	// required: true
-	CloneAddr    string `json:"clone_addr" binding:"Required"`
-	AuthUsername string `json:"auth_username"`
-	AuthPassword string `json:"auth_password"`
-	AuthToken    string `json:"auth_token"`
+	CloneAddr             string `json:"clone_addr" binding:"Required"`
+	CloneAddrEncrypted    string `json:"clone_addr_encrypted,omitempty"`
+	AuthUsername          string `json:"auth_username"`
+	AuthPassword          string `json:"auth_password,omitempty"`
+	AuthPasswordEncrypted string `json:"auth_password_encrypted,omitempty"`
+	AuthToken             string `json:"auth_token,omitempty"`
+	AuthTokenEncrypted    string `json:"auth_token_encrypted,omitempty"`
 	// required: true
 	UID int `json:"uid" binding:"Required"`
 	// required: true

modules/migrations/dump.go

@@ -13,6 +13,7 @@ import (
 	"os"
 	"path/filepath"
 	"strconv"
+	"strings"
 	"time"
 
 	"code.gitea.io/gitea/models"
@@ -563,8 +564,42 @@ func DumpRepository(ctx context.Context, baseDir, ownerName string, opts base.Mi
 	return nil
 }
 
+func updateOptionsUnits(opts *base.MigrateOptions, units []string) {
+	if len(units) == 0 {
+		opts.Wiki = true
+		opts.Issues = true
+		opts.Milestones = true
+		opts.Labels = true
+		opts.Releases = true
+		opts.Comments = true
+		opts.PullRequests = true
+		opts.ReleaseAssets = true
+	} else {
+		for _, unit := range units {
+			switch strings.ToLower(unit) {
+			case "wiki":
+				opts.Wiki = true
+			case "issues":
+				opts.Issues = true
+			case "milestones":
+				opts.Milestones = true
+			case "labels":
+				opts.Labels = true
+			case "releases":
+				opts.Releases = true
+			case "release_assets":
+				opts.ReleaseAssets = true
+			case "comments":
+				opts.Comments = true
+			case "pull_requests":
+				opts.PullRequests = true
+			}
+		}
+	}
+}
+
 // RestoreRepository restore a repository from the disk directory
-func RestoreRepository(ctx context.Context, baseDir string, ownerName, repoName string) error {
+func RestoreRepository(ctx context.Context, baseDir string, ownerName, repoName string, units []string) error {
 	doer, err := models.GetAdminUser()
 	if err != nil {
 		return err
@@ -580,17 +615,12 @@ func RestoreRepository(ctx context.Context, baseDir string, ownerName, repoName
 	}
 	tp, _ := strconv.Atoi(opts["service_type"])
 
-	if err = migrateRepository(downloader, uploader, base.MigrateOptions{
-		Wiki:           true,
-		Issues:         true,
-		Milestones:     true,
-		Labels:         true,
-		Releases:       true,
-		Comments:       true,
-		PullRequests:   true,
-		ReleaseAssets:  true,
+	var migrateOpts = base.MigrateOptions{
 		GitServiceType: structs.GitServiceType(tp),
-	}); err != nil {
+	}
+	updateOptionsUnits(&migrateOpts, units)
+
+	if err = migrateRepository(downloader, uploader, migrateOpts); err != nil {
 		if err1 := uploader.Rollback(); err1 != nil {
 			log.Error("rollback failed: %v", err1)
 		}

modules/migrations/gitea_uploader.go

@@ -248,14 +248,16 @@ func (g *GiteaLocalUploader) CreateReleases(releases ...*base.Release) error {
 			rel.OriginalAuthorID = release.PublisherID
 		}
 
-		// calc NumCommits
-		commit, err := g.gitRepo.GetCommit(rel.TagName)
-		if err != nil {
-			return fmt.Errorf("GetCommit: %v", err)
-		}
-		rel.NumCommits, err = commit.CommitsCount()
-		if err != nil {
-			return fmt.Errorf("CommitsCount: %v", err)
+		// calc NumCommits if no draft
+		if !release.Draft {
+			commit, err := g.gitRepo.GetCommit(rel.TagName)
+			if err != nil {
+				return fmt.Errorf("GetCommit: %v", err)
+			}
+			rel.NumCommits, err = commit.CommitsCount()
+			if err != nil {
+				return fmt.Errorf("CommitsCount: %v", err)
+			}
 		}
 
 		for _, asset := range release.Assets {
@@ -268,9 +270,10 @@ func (g *GiteaLocalUploader) CreateReleases(releases ...*base.Release) error {
 			}
 
 			// download attachment
-			err = func() error {
+			err := func() error {
 				// asset.DownloadURL maybe a local file
 				var rc io.ReadCloser
+				var err error
 				if asset.DownloadURL == nil {
 					rc, err = asset.DownloadFunc()
 					if err != nil {
@@ -849,6 +852,7 @@ func (g *GiteaLocalUploader) CreateReviews(reviews ...*base.Review) error {
 // Rollback when migrating failed, this will rollback all the changes.
 func (g *GiteaLocalUploader) Rollback() error {
 	if g.repo != nil && g.repo.ID > 0 {
+		g.gitRepo.Close()
 		if err := models.DeleteRepository(g.doer, g.repo.OwnerID, g.repo.ID); err != nil {
 			return err
 		}

modules/migrations/github.go

@@ -264,34 +264,29 @@ func (g *GithubDownloaderV3) GetLabels() ([]*base.Label, error) {
 }
 
 func (g *GithubDownloaderV3) convertGithubRelease(rel *github.RepositoryRelease) *base.Release {
-	var (
-		name string
-		desc string
-	)
-	if rel.Body != nil {
-		desc = *rel.Body
-	}
-	if rel.Name != nil {
-		name = *rel.Name
-	}
-
-	var email string
-	if rel.Author.Email != nil {
-		email = *rel.Author.Email
-	}
-
 	r := &base.Release{
 		TagName:         *rel.TagName,
 		TargetCommitish: *rel.TargetCommitish,
-		Name:            name,
-		Body:            desc,
 		Draft:           *rel.Draft,
 		Prerelease:      *rel.Prerelease,
 		Created:         rel.CreatedAt.Time,
 		PublisherID:     *rel.Author.ID,
 		PublisherName:   *rel.Author.Login,
-		PublisherEmail:  email,
-		Published:       rel.PublishedAt.Time,
+	}
+
+	if rel.Body != nil {
+		r.Body = *rel.Body
+	}
+	if rel.Name != nil {
+		r.Name = *rel.Name
+	}
+
+	if rel.Author.Email != nil {
+		r.PublisherEmail = *rel.Author.Email
+	}
+
+	if rel.PublishedAt != nil {
+		r.Published = rel.PublishedAt.Time
 	}
 
 	for _, asset := range rel.Assets {
@@ -306,18 +301,17 @@ func (g *GithubDownloaderV3) convertGithubRelease(rel *github.RepositoryRelease)
 			Updated:       asset.UpdatedAt.Time,
 			DownloadFunc: func() (io.ReadCloser, error) {
 				g.sleep()
-				asset, redir, err := g.client.Repositories.DownloadReleaseAsset(g.ctx, g.repoOwner, g.repoName, assetID, nil)
+				asset, redirectURL, err := g.client.Repositories.DownloadReleaseAsset(g.ctx, g.repoOwner, g.repoName, assetID, nil)
 				if err != nil {
 					return nil, err
 				}
-				err = g.RefreshRate()
-				if err != nil {
+				if err := g.RefreshRate(); err != nil {
 					log.Error("g.client.RateLimits: %s", err)
 				}
 				if asset == nil {
-					if redir != "" {
+					if redirectURL != "" {
 						g.sleep()
-						req, err := http.NewRequestWithContext(g.ctx, "GET", redir, nil)
+						req, err := http.NewRequestWithContext(g.ctx, "GET", redirectURL, nil)
 						if err != nil {
 							return nil, err
 						}

modules/notification/mail/mail.go

@@ -54,7 +54,6 @@ func (m *mailNotifier) NotifyNewIssue(issue *models.Issue, mentions []*models.Us
 
 func (m *mailNotifier) NotifyIssueChangeStatus(doer *models.User, issue *models.Issue, actionComment *models.Comment, isClosed bool) {
 	var actionType models.ActionType
-	issue.Content = ""
 	if issue.IsPull {
 		if isClosed {
 			actionType = models.ActionClosePullRequest
@@ -120,7 +119,6 @@ func (m *mailNotifier) NotifyMergePullRequest(pr *models.PullRequest, doer *mode
 		log.Error("pr.LoadIssue: %v", err)
 		return
 	}
-	pr.Issue.Content = ""
 	if err := mailer.MailParticipants(pr.Issue, doer, models.ActionMergePullRequest, nil); err != nil {
 		log.Error("MailParticipants: %v", err)
 	}
@@ -147,7 +145,6 @@ func (m *mailNotifier) NotifyPullRequestPushCommits(doer *models.User, pr *model
 	if err := comment.LoadPushCommits(); err != nil {
 		log.Error("comment.LoadPushCommits: %v", err)
 	}
-	comment.Content = ""
 
 	m.NotifyCreateIssueComment(doer, comment.Issue.Repo, comment.Issue, comment, nil)
 }

modules/private/restore_repo.go

@@ -0,0 +1,60 @@
+// Copyright 2020 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package private
+
+import (
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"time"
+
+	"code.gitea.io/gitea/modules/setting"
+	jsoniter "github.com/json-iterator/go"
+)
+
+// RestoreParams structure holds a data for restore repository
+type RestoreParams struct {
+	RepoDir   string
+	OwnerName string
+	RepoName  string
+	Units     []string
+}
+
+// RestoreRepo calls the internal RestoreRepo function
+func RestoreRepo(repoDir, ownerName, repoName string, units []string) (int, string) {
+	reqURL := setting.LocalURL + "api/internal/restore_repo"
+
+	req := newInternalRequest(reqURL, "POST")
+	req.SetTimeout(3*time.Second, 0) // since the request will spend much time, don't timeout
+	req = req.Header("Content-Type", "application/json")
+	json := jsoniter.ConfigCompatibleWithStandardLibrary
+	jsonBytes, _ := json.Marshal(RestoreParams{
+		RepoDir:   repoDir,
+		OwnerName: ownerName,
+		RepoName:  repoName,
+		Units:     units,
+	})
+	req.Body(jsonBytes)
+	resp, err := req.Response()
+	if err != nil {
+		return http.StatusInternalServerError, fmt.Sprintf("Unable to contact gitea: %v, could you confirm it's running?", err.Error())
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != 200 {
+		var ret = struct {
+			Err string `json:"err"`
+		}{}
+		body, err := ioutil.ReadAll(resp.Body)
+		if err != nil {
+			return http.StatusInternalServerError, fmt.Sprintf("Response body error: %v", err.Error())
+		}
+		if err := json.Unmarshal(body, &ret); err != nil {
+			return http.StatusInternalServerError, fmt.Sprintf("Response body Unmarshal error: %v", err.Error())
+		}
+	}
+
+	return http.StatusOK, fmt.Sprintf("Restore repo %s/%s successfully", ownerName, repoName)
+}

modules/queue/manager.go

@@ -198,17 +198,20 @@ func (m *Manager) FlushAll(baseCtx context.Context, timeout time.Duration) error
 					wg.Done()
 				}(mq)
 			} else {
-				log.Debug("Queue: %s is non-empty but is not flushable - adding 100 millisecond wait", mq.Name)
-				go func() {
-					<-time.After(100 * time.Millisecond)
-					wg.Done()
-				}()
+				log.Debug("Queue: %s is non-empty but is not flushable", mq.Name)
+				wg.Done()
 			}
-
 		}
 		if allEmpty {
+			log.Debug("All queues are empty")
 			break
 		}
+		// Ensure there are always at least 100ms between loops but not more if we've actually been doing some flushign
+		// but don't delay cancellation here.
+		select {
+		case <-ctx.Done():
+		case <-time.After(100 * time.Millisecond):
+		}
 		wg.Wait()
 	}
 	return nil

modules/references/references.go

@@ -5,6 +5,7 @@
 package references
 
 import (
+	"bytes"
 	"net/url"
 	"regexp"
 	"strconv"
@@ -14,6 +15,8 @@ import (
 	"code.gitea.io/gitea/modules/log"
 	"code.gitea.io/gitea/modules/markup/mdstripper"
 	"code.gitea.io/gitea/modules/setting"
+
+	"github.com/yuin/goldmark/util"
 )
 
 var (
@@ -321,7 +324,7 @@ func FindRenderizableReferenceNumeric(content string, prOnly bool) (bool, *Rende
 			return false, nil
 		}
 	}
-	r := getCrossReference([]byte(content), match[2], match[3], false, prOnly)
+	r := getCrossReference(util.StringToReadOnlyBytes(content), match[2], match[3], false, prOnly)
 	if r == nil {
 		return false, nil
 	}
@@ -465,17 +468,16 @@ func findAllIssueReferencesBytes(content []byte, links []string) []*rawReference
 }
 
 func getCrossReference(content []byte, start, end int, fromLink bool, prOnly bool) *rawReference {
-	refid := string(content[start:end])
-	sep := strings.IndexAny(refid, "#!")
+	sep := bytes.IndexAny(content[start:end], "#!")
 	if sep < 0 {
 		return nil
 	}
-	isPull := refid[sep] == '!'
+	isPull := content[start+sep] == '!'
 	if prOnly && !isPull {
 		return nil
 	}
-	repo := refid[:sep]
-	issue := refid[sep+1:]
+	repo := string(content[start : start+sep])
+	issue := string(content[start+sep+1 : end])
 	index, err := strconv.ParseInt(issue, 10, 64)
 	if err != nil {
 		return nil

modules/setting/setting.go

@@ -117,6 +117,8 @@ var (
 	GracefulRestartable  bool
 	GracefulHammerTime   time.Duration
 	StartupTimeout       time.Duration
+	PerWriteTimeout      = 30 * time.Second
+	PerWritePerKbTimeout = 10 * time.Second
 	StaticURLPrefix      string
 	AbsoluteAssetURL     string
 
@@ -147,18 +149,22 @@ var (
 		TrustedUserCAKeys              []string          `ini:"SSH_TRUSTED_USER_CA_KEYS"`
 		TrustedUserCAKeysFile          string            `ini:"SSH_TRUSTED_USER_CA_KEYS_FILENAME"`
 		TrustedUserCAKeysParsed        []gossh.PublicKey `ini:"-"`
+		PerWriteTimeout                time.Duration     `ini:"SSH_PER_WRITE_TIMEOUT"`
+		PerWritePerKbTimeout           time.Duration     `ini:"SSH_PER_WRITE_PER_KB_TIMEOUT"`
 	}{
-		Disabled:            false,
-		StartBuiltinServer:  false,
-		Domain:              "",
-		Port:                22,
-		ServerCiphers:       []string{"aes128-ctr", "aes192-ctr", "aes256-ctr", "aes128-gcm@openssh.com", "arcfour256", "arcfour128"},
-		ServerKeyExchanges:  []string{"diffie-hellman-group1-sha1", "diffie-hellman-group14-sha1", "ecdh-sha2-nistp256", "ecdh-sha2-nistp384", "ecdh-sha2-nistp521", "curve25519-sha256@libssh.org"},
-		ServerMACs:          []string{"hmac-sha2-256-etm@openssh.com", "hmac-sha2-256", "hmac-sha1", "hmac-sha1-96"},
-		KeygenPath:          "ssh-keygen",
-		MinimumKeySizeCheck: true,
-		MinimumKeySizes:     map[string]int{"ed25519": 256, "ed25519-sk": 256, "ecdsa": 256, "ecdsa-sk": 256, "rsa": 2048},
-		ServerHostKeys:      []string{"ssh/gitea.rsa", "ssh/gogs.rsa"},
+		Disabled:             false,
+		StartBuiltinServer:   false,
+		Domain:               "",
+		Port:                 22,
+		ServerCiphers:        []string{"aes128-ctr", "aes192-ctr", "aes256-ctr", "aes128-gcm@openssh.com", "arcfour256", "arcfour128"},
+		ServerKeyExchanges:   []string{"diffie-hellman-group1-sha1", "diffie-hellman-group14-sha1", "ecdh-sha2-nistp256", "ecdh-sha2-nistp384", "ecdh-sha2-nistp521", "curve25519-sha256@libssh.org"},
+		ServerMACs:           []string{"hmac-sha2-256-etm@openssh.com", "hmac-sha2-256", "hmac-sha1", "hmac-sha1-96"},
+		KeygenPath:           "ssh-keygen",
+		MinimumKeySizeCheck:  true,
+		MinimumKeySizes:      map[string]int{"ed25519": 256, "ed25519-sk": 256, "ecdsa": 256, "ecdsa-sk": 256, "rsa": 2048},
+		ServerHostKeys:       []string{"ssh/gitea.rsa", "ssh/gogs.rsa"},
+		PerWriteTimeout:      PerWriteTimeout,
+		PerWritePerKbTimeout: PerWritePerKbTimeout,
 	}
 
 	// Security settings
@@ -607,6 +613,8 @@ func NewContext() {
 	GracefulRestartable = sec.Key("ALLOW_GRACEFUL_RESTARTS").MustBool(true)
 	GracefulHammerTime = sec.Key("GRACEFUL_HAMMER_TIME").MustDuration(60 * time.Second)
 	StartupTimeout = sec.Key("STARTUP_TIMEOUT").MustDuration(0 * time.Second)
+	PerWriteTimeout = sec.Key("PER_WRITE_TIMEOUT").MustDuration(PerWriteTimeout)
+	PerWritePerKbTimeout = sec.Key("PER_WRITE_PER_KB_TIMEOUT").MustDuration(PerWritePerKbTimeout)
 
 	defaultAppURL := string(Protocol) + "://" + Domain
 	if (Protocol == HTTP && HTTPPort != "80") || (Protocol == HTTPS && HTTPPort != "443") {
@@ -772,6 +780,8 @@ func NewContext() {
 	}
 
 	SSH.ExposeAnonymous = sec.Key("SSH_EXPOSE_ANONYMOUS").MustBool(false)
+	SSH.PerWriteTimeout = sec.Key("SSH_PER_WRITE_TIMEOUT").MustDuration(PerWriteTimeout)
+	SSH.PerWritePerKbTimeout = sec.Key("SSH_PER_WRITE_PER_KB_TIMEOUT").MustDuration(PerWritePerKbTimeout)
 
 	if err = Cfg.Section("oauth2").MapTo(&OAuth2); err != nil {
 		log.Fatal("Failed to OAuth2 settings: %v", err)

modules/ssh/ssh_graceful.go

@@ -7,12 +7,15 @@ package ssh
 import (
 	"code.gitea.io/gitea/modules/graceful"
 	"code.gitea.io/gitea/modules/log"
+	"code.gitea.io/gitea/modules/setting"
 
 	"github.com/gliderlabs/ssh"
 )
 
 func listen(server *ssh.Server) {
 	gracefulServer := graceful.NewServer("tcp", server.Addr, "SSH")
+	gracefulServer.PerWriteTimeout = setting.SSH.PerWriteTimeout
+	gracefulServer.PerWritePerKbTimeout = setting.SSH.PerWritePerKbTimeout
 
 	err := gracefulServer.ListenAndServe(server.Serve)
 	if err != nil {

modules/storage/local.go

@@ -96,7 +96,7 @@ func (l *LocalStorage) Save(path string, r io.Reader, size int64) (int64, error)
 		return 0, err
 	}
 
-	if err := os.Rename(tmp.Name(), p); err != nil {
+	if err := util.Rename(tmp.Name(), p); err != nil {
 		return 0, err
 	}
 

modules/structs/org.go

@@ -31,6 +31,8 @@ type CreateOrgOption struct {
 	RepoAdminChangeTeamAccess bool   `json:"repo_admin_change_team_access"`
 }
 
+// TODO: make EditOrgOption fields optional after https://gitea.com/go-chi/binding/pulls/5 got merged
+
 // EditOrgOption options for editing an organization
 type EditOrgOption struct {
 	FullName    string `json:"full_name"`
@@ -40,5 +42,5 @@ type EditOrgOption struct {
 	// possible values are `public`, `limited` or `private`
 	// enum: public,limited,private
 	Visibility                string `json:"visibility" binding:"In(,public,limited,private)"`
-	RepoAdminChangeTeamAccess bool   `json:"repo_admin_change_team_access"`
+	RepoAdminChangeTeamAccess *bool  `json:"repo_admin_change_team_access"`
 }

modules/task/task.go

@@ -13,8 +13,11 @@ import (
 	"code.gitea.io/gitea/modules/migrations/base"
 	"code.gitea.io/gitea/modules/queue"
 	repo_module "code.gitea.io/gitea/modules/repository"
+	"code.gitea.io/gitea/modules/secret"
+	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/modules/structs"
 	"code.gitea.io/gitea/modules/timeutil"
+	"code.gitea.io/gitea/modules/util"
 	jsoniter "github.com/json-iterator/go"
 )
 
@@ -65,6 +68,24 @@ func MigrateRepository(doer, u *models.User, opts base.MigrateOptions) error {
 
 // CreateMigrateTask creates a migrate task
 func CreateMigrateTask(doer, u *models.User, opts base.MigrateOptions) (*models.Task, error) {
+	// encrypt credentials for persistence
+	var err error
+	opts.CloneAddrEncrypted, err = secret.EncryptSecret(setting.SecretKey, opts.CloneAddr)
+	if err != nil {
+		return nil, err
+	}
+	opts.CloneAddr = util.SanitizeURLCredentials(opts.CloneAddr, true)
+	opts.AuthPasswordEncrypted, err = secret.EncryptSecret(setting.SecretKey, opts.AuthPassword)
+	if err != nil {
+		return nil, err
+	}
+	opts.AuthPassword = ""
+	opts.AuthTokenEncrypted, err = secret.EncryptSecret(setting.SecretKey, opts.AuthToken)
+	if err != nil {
+		return nil, err
+	}
+	opts.AuthToken = ""
+
 	json := jsoniter.ConfigCompatibleWithStandardLibrary
 	bs, err := json.Marshal(&opts)
 	if err != nil {

modules/util/remove.go

@@ -33,7 +33,7 @@ func Remove(name string) error {
 	return err
 }
 
-// RemoveAll removes the named file or (empty) directory with at most 5 attempts.Remove
+// RemoveAll removes the named file or (empty) directory with at most 5 attempts.
 func RemoveAll(name string) error {
 	var err error
 	for i := 0; i < 5; i++ {
@@ -55,3 +55,30 @@ func RemoveAll(name string) error {
 	}
 	return err
 }
+
+// Rename renames (moves) oldpath to newpath with at most 5 attempts.
+func Rename(oldpath, newpath string) error {
+	var err error
+	for i := 0; i < 5; i++ {
+		err = os.Rename(oldpath, newpath)
+		if err == nil {
+			break
+		}
+		unwrapped := err.(*os.PathError).Err
+		if unwrapped == syscall.EBUSY || unwrapped == syscall.ENOTEMPTY || unwrapped == syscall.EPERM || unwrapped == syscall.EMFILE || unwrapped == syscall.ENFILE {
+			// try again
+			<-time.After(100 * time.Millisecond)
+			continue
+		}
+
+		if i == 0 && os.IsNotExist(err) {
+			return err
+		}
+
+		if unwrapped == syscall.ENOENT {
+			// it's already gone
+			return nil
+		}
+	}
+	return err
+}

modules/web/middleware/cookie.go

@@ -149,7 +149,7 @@ func SetCookie(resp http.ResponseWriter, name string, value string, others ...in
 	if len(others) > 2 {
 		if v, ok := others[2].(string); ok && len(v) > 0 {
 			cookie.Domain = v
-		} else if v, ok := others[1].(func(*http.Cookie)); ok {
+		} else if v, ok := others[2].(func(*http.Cookie)); ok {
 			v(&cookie)
 		}
 	}
@@ -170,7 +170,7 @@ func SetCookie(resp http.ResponseWriter, name string, value string, others ...in
 	if len(others) > 4 {
 		if v, ok := others[4].(bool); ok && v {
 			cookie.HttpOnly = true
-		} else if v, ok := others[1].(func(*http.Cookie)); ok {
+		} else if v, ok := others[4].(func(*http.Cookie)); ok {
 			v(&cookie)
 		}
 	}
@@ -179,7 +179,7 @@ func SetCookie(resp http.ResponseWriter, name string, value string, others ...in
 		if v, ok := others[5].(time.Time); ok {
 			cookie.Expires = v
 			cookie.RawExpires = v.Format(time.UnixDate)
-		} else if v, ok := others[1].(func(*http.Cookie)); ok {
+		} else if v, ok := others[5].(func(*http.Cookie)); ok {
 			v(&cookie)
 		}
 	}

options/locale/locale_en-US.ini

@@ -96,6 +96,8 @@ error404 = The page you are trying to reach either <strong>does not exist</stron
 [error]
 occurred = An error has occurred
 report_message = If you are sure this is a Gitea bug, please search for issue on <a href="https://github.com/go-gitea/gitea/issues">GitHub</a> and open new issue if necessary.
+missing_csrf = Bad Request: no CSRF token present
+invalid_csrf = Bad Request: Invalid CSRF token
 
 [startpage]
 app_desc = A painless, self-hosted Git service
@@ -719,6 +721,9 @@ mirror_address_desc = Put any required credentials in the Clone Authorization se
 mirror_address_url_invalid = The provided url is invalid. You must escape all components of the url correctly.
 mirror_address_protocol_invalid = The provided url is invalid. Only http(s):// or git:// locations can be mirrored from.
 mirror_last_synced = Last Synchronized
+mirror_password_placeholder = (Unchanged)
+mirror_password_blank_placeholder = (Unset)
+mirror_password_help = Change the username to erase a stored password.
 watchers = Watchers
 stargazers = Stargazers
 forks = Forks
@@ -2281,6 +2286,7 @@ auths.allowed_domains_helper = Leave empty to allow all domains. Separate multip
 auths.enable_tls = Enable TLS Encryption
 auths.skip_tls_verify = Skip TLS Verify
 auths.pam_service_name = PAM Service Name
+auths.pam_email_domain = PAM Email Domain (optional)
 auths.oauth2_provider = OAuth2 Provider
 auths.oauth2_icon_url = Icon URL
 auths.oauth2_clientID = Client ID (Key)

routers/admin/auths.go

@@ -239,6 +239,7 @@ func NewAuthSourcePost(ctx *context.Context) {
 	case models.LoginPAM:
 		config = &models.PAMConfig{
 			ServiceName: form.PAMServiceName,
+			EmailDomain: form.PAMEmailDomain,
 		}
 	case models.LoginOAuth2:
 		config = parseOAuth2Config(form)
@@ -346,6 +347,7 @@ func EditAuthSourcePost(ctx *context.Context) {
 	case models.LoginPAM:
 		config = &models.PAMConfig{
 			ServiceName: form.PAMServiceName,
+			EmailDomain: form.PAMEmailDomain,
 		}
 	case models.LoginOAuth2:
 		config = parseOAuth2Config(form)

routers/admin/repos.go

@@ -46,6 +46,10 @@ func DeleteRepo(ctx *context.Context) {
 		return
 	}
 
+	if ctx.Repo != nil && ctx.Repo.GitRepo != nil && ctx.Repo.Repository != nil && ctx.Repo.Repository.ID == repo.ID {
+		ctx.Repo.GitRepo.Close()
+	}
+
 	if err := repo_service.DeleteRepository(ctx.User, repo); err != nil {
 		ctx.ServerError("DeleteRepository", err)
 		return

routers/api/v1/api.go

@@ -557,6 +557,7 @@ func Routes() *web.Route {
 		Gclifetime:     setting.SessionConfig.Gclifetime,
 		Maxlifetime:    setting.SessionConfig.Maxlifetime,
 		Secure:         setting.SessionConfig.Secure,
+		SameSite:       setting.SessionConfig.SameSite,
 		Domain:         setting.SessionConfig.Domain,
 	}))
 	m.Use(securityHeaders())
@@ -892,7 +893,7 @@ func Routes() *web.Route {
 						Post(reqToken(), mustNotBeArchived, bind(api.CreatePullRequestOption{}), repo.CreatePullRequest)
 					m.Group("/{index}", func() {
 						m.Combo("").Get(repo.GetPullRequest).
-							Patch(reqToken(), reqRepoWriter(models.UnitTypePullRequests), bind(api.EditPullRequestOption{}), repo.EditPullRequest)
+							Patch(reqToken(), bind(api.EditPullRequestOption{}), repo.EditPullRequest)
 						m.Get(".diff", repo.DownloadPullDiff)
 						m.Get(".patch", repo.DownloadPullPatch)
 						m.Post("/update", reqToken(), repo.UpdatePullRequest)
@@ -985,10 +986,10 @@ func Routes() *web.Route {
 					Delete(reqToken(), reqOrgMembership(), org.ConcealMember)
 			})
 			m.Group("/teams", func() {
-				m.Combo("", reqToken()).Get(org.ListTeams).
-					Post(reqOrgOwnership(), bind(api.CreateTeamOption{}), org.CreateTeam)
+				m.Get("", org.ListTeams)
+				m.Post("", reqOrgOwnership(), bind(api.CreateTeamOption{}), org.CreateTeam)
 				m.Get("/search", org.SearchTeam)
-			}, reqOrgMembership())
+			}, reqToken(), reqOrgMembership())
 			m.Group("/labels", func() {
 				m.Get("", org.ListLabels)
 				m.Post("", reqToken(), reqOrgOwnership(), bind(api.CreateLabelOption{}), org.CreateLabel)

routers/api/v1/org/org.go

@@ -264,7 +264,13 @@ func Edit(ctx *context.APIContext) {
 	if form.Visibility != "" {
 		org.Visibility = api.VisibilityModes[form.Visibility]
 	}
-	if err := models.UpdateUserCols(org, "full_name", "description", "website", "location", "visibility"); err != nil {
+	if form.RepoAdminChangeTeamAccess != nil {
+		org.RepoAdminChangeTeamAccess = *form.RepoAdminChangeTeamAccess
+	}
+	if err := models.UpdateUserCols(org,
+		"full_name", "description", "website", "location",
+		"visibility", "repo_admin_change_team_access",
+	); err != nil {
 		ctx.Error(http.StatusInternalServerError, "EditOrganization", err)
 		return
 	}

routers/api/v1/repo/branch.go

@@ -6,6 +6,7 @@
 package repo
 
 import (
+	"errors"
 	"fmt"
 	"net/http"
 
@@ -13,7 +14,6 @@ import (
 	"code.gitea.io/gitea/modules/context"
 	"code.gitea.io/gitea/modules/convert"
 	"code.gitea.io/gitea/modules/git"
-	"code.gitea.io/gitea/modules/log"
 	repo_module "code.gitea.io/gitea/modules/repository"
 	api "code.gitea.io/gitea/modules/structs"
 	"code.gitea.io/gitea/modules/web"
@@ -117,62 +117,20 @@ func DeleteBranch(ctx *context.APIContext) {
 
 	branchName := ctx.Params("*")
 
-	if ctx.Repo.Repository.DefaultBranch == branchName {
-		ctx.Error(http.StatusForbidden, "DefaultBranch", fmt.Errorf("can not delete default branch"))
-		return
-	}
-
-	isProtected, err := ctx.Repo.Repository.IsProtectedBranch(branchName, ctx.User)
-	if err != nil {
-		ctx.InternalServerError(err)
-		return
-	}
-	if isProtected {
-		ctx.Error(http.StatusForbidden, "IsProtectedBranch", fmt.Errorf("branch protected"))
-		return
-	}
-
-	branch, err := repo_module.GetBranch(ctx.Repo.Repository, branchName)
-	if err != nil {
-		if git.IsErrBranchNotExist(err) {
+	if err := repo_service.DeleteBranch(ctx.User, ctx.Repo.Repository, ctx.Repo.GitRepo, branchName); err != nil {
+		switch {
+		case git.IsErrBranchNotExist(err):
 			ctx.NotFound(err)
-		} else {
-			ctx.Error(http.StatusInternalServerError, "GetBranch", err)
+		case errors.Is(err, repo_service.ErrBranchIsDefault):
+			ctx.Error(http.StatusForbidden, "DefaultBranch", fmt.Errorf("can not delete default branch"))
+		case errors.Is(err, repo_service.ErrBranchIsProtected):
+			ctx.Error(http.StatusForbidden, "IsProtectedBranch", fmt.Errorf("branch protected"))
+		default:
+			ctx.Error(http.StatusInternalServerError, "DeleteBranch", err)
 		}
 		return
 	}
 
-	c, err := branch.GetCommit()
-	if err != nil {
-		ctx.Error(http.StatusInternalServerError, "GetCommit", err)
-		return
-	}
-
-	if err := ctx.Repo.GitRepo.DeleteBranch(branchName, git.DeleteBranchOptions{
-		Force: true,
-	}); err != nil {
-		ctx.Error(http.StatusInternalServerError, "DeleteBranch", err)
-		return
-	}
-
-	// Don't return error below this
-	if err := repo_service.PushUpdate(
-		&repo_module.PushUpdateOptions{
-			RefFullName:  git.BranchPrefix + branchName,
-			OldCommitID:  c.ID.String(),
-			NewCommitID:  git.EmptySHA,
-			PusherID:     ctx.User.ID,
-			PusherName:   ctx.User.Name,
-			RepoUserName: ctx.Repo.Owner.Name,
-			RepoName:     ctx.Repo.Repository.Name,
-		}); err != nil {
-		log.Error("Update: %v", err)
-	}
-
-	if err := ctx.Repo.Repository.AddDeletedBranch(branchName, c.ID.String(), ctx.User.ID); err != nil {
-		log.Warn("AddDeletedBranch: %v", err)
-	}
-
 	ctx.Status(http.StatusNoContent)
 }
 

routers/api/v1/repo/repo.go

@@ -885,6 +885,10 @@ func Delete(ctx *context.APIContext) {
 		return
 	}
 
+	if ctx.Repo.GitRepo != nil {
+		ctx.Repo.GitRepo.Close()
+	}
+
 	if err := repo_service.DeleteRepository(ctx.User, repo); err != nil {
 		ctx.Error(http.StatusInternalServerError, "DeleteRepository", err)
 		return

routers/api/v1/user/helper.go

@@ -17,7 +17,7 @@ func GetUserByParamsName(ctx *context.APIContext, name string) *models.User {
 	user, err := models.GetUserByName(username)
 	if err != nil {
 		if models.IsErrUserNotExist(err) {
-			if redirectUserID, err := models.LookupUserRedirect(username); err == nil {
+			if redirectUserID, err2 := models.LookupUserRedirect(username); err2 == nil {
 				context.RedirectToUser(ctx.Context, username, redirectUserID)
 			} else {
 				ctx.NotFound("GetUserByName", err)

routers/api/v1/utils/utils.go

@@ -55,7 +55,7 @@ func parseTime(value string) (int64, error) {
 // prepareQueryArg unescape and trim a query arg
 func prepareQueryArg(ctx *context.APIContext, name string) (value string, err error) {
 	value, err = url.PathUnescape(ctx.Query(name))
-	value = strings.Trim(value, " ")
+	value = strings.TrimSpace(value)
 	return
 }
 

routers/install.go

@@ -22,6 +22,7 @@ import (
 	"code.gitea.io/gitea/modules/log"
 	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/modules/templates"
+	"code.gitea.io/gitea/modules/translation"
 	"code.gitea.io/gitea/modules/user"
 	"code.gitea.io/gitea/modules/util"
 	"code.gitea.io/gitea/modules/web"
@@ -61,6 +62,8 @@ func InstallInit(next http.Handler) http.Handler {
 				"DbOptions":     setting.SupportedDatabases,
 				"i18n":          locale,
 				"Language":      locale.Language(),
+				"Lang":          locale.Language(),
+				"AllLangs":      translation.AllLangs(),
 				"CurrentURL":    setting.AppSubURL + req.URL.RequestURI(),
 				"PageStartTime": startTime,
 				"TmplLoadTimes": func() string {
@@ -69,6 +72,12 @@ func InstallInit(next http.Handler) http.Handler {
 				"PasswordHashAlgorithms": models.AvailableHashAlgorithms,
 			},
 		}
+		for _, lang := range translation.AllLangs() {
+			if lang.Lang == locale.Language() {
+				ctx.Data["LangName"] = lang.Name
+				break
+			}
+		}
 		ctx.Req = context.WithContext(req, &ctx)
 		next.ServeHTTP(resp, ctx.Req)
 	})

routers/org/setting.go

@@ -51,6 +51,7 @@ func SettingsPost(ctx *context.Context) {
 	}
 
 	org := ctx.Org.Organization
+	nameChanged := org.Name != form.Name
 
 	// Check if organization name has been changed.
 	if org.LowerName != strings.ToLower(form.Name) {
@@ -74,7 +75,9 @@ func SettingsPost(ctx *context.Context) {
 		// reset ctx.org.OrgLink with new name
 		ctx.Org.OrgLink = setting.AppSubURL + "/org/" + form.Name
 		log.Trace("Organization name changed: %s -> %s", org.Name, form.Name)
+		nameChanged = false
 	}
+
 	// In case it's just a case change.
 	org.Name = form.Name
 	org.LowerName = strings.ToLower(form.Name)
@@ -104,11 +107,17 @@ func SettingsPost(ctx *context.Context) {
 			return
 		}
 		for _, repo := range org.Repos {
+			repo.OwnerName = org.Name
 			if err := models.UpdateRepository(repo, true); err != nil {
 				ctx.ServerError("UpdateRepository", err)
 				return
 			}
 		}
+	} else if nameChanged {
+		if err := models.UpdateRepositoryOwnerNames(org.ID, org.Name); err != nil {
+			ctx.ServerError("UpdateRepository", err)
+			return
+		}
 	}
 
 	log.Trace("Organization setting updated: %s", org.Name)

routers/private/internal.go

@@ -69,6 +69,7 @@ func Routes() *web.Route {
 	r.Post("/manager/add-logger", bind(private.LoggerOptions{}), AddLogger)
 	r.Post("/manager/remove-logger/{group}/{name}", RemoveLogger)
 	r.Post("/mail/send", SendEmail)
+	r.Post("/restore_repo", RestoreRepo)
 
 	return r
 }

routers/private/restore_repo.go

@@ -0,0 +1,51 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package private
+
+import (
+	"io/ioutil"
+
+	myCtx "code.gitea.io/gitea/modules/context"
+	"code.gitea.io/gitea/modules/migrations"
+	jsoniter "github.com/json-iterator/go"
+)
+
+// RestoreRepo restore a repository from data
+func RestoreRepo(ctx *myCtx.PrivateContext) {
+	json := jsoniter.ConfigCompatibleWithStandardLibrary
+	bs, err := ioutil.ReadAll(ctx.Req.Body)
+	if err != nil {
+		ctx.JSON(500, map[string]string{
+			"err": err.Error(),
+		})
+		return
+	}
+	var params = struct {
+		RepoDir   string
+		OwnerName string
+		RepoName  string
+		Units     []string
+	}{}
+	if err = json.Unmarshal(bs, &params); err != nil {
+		ctx.JSON(500, map[string]string{
+			"err": err.Error(),
+		})
+		return
+	}
+
+	if err := migrations.RestoreRepository(
+		ctx.Req.Context(),
+		params.RepoDir,
+		params.OwnerName,
+		params.RepoName,
+		params.Units,
+	); err != nil {
+		ctx.JSON(500, map[string]string{
+			"err": err.Error(),
+		})
+	} else {
+		ctx.Status(200)
+	}
+}

routers/repo/branch.go

@@ -6,6 +6,7 @@
 package repo
 
 import (
+	"errors"
 	"fmt"
 	"strings"
 
@@ -82,34 +83,23 @@ func Branches(ctx *context.Context) {
 func DeleteBranchPost(ctx *context.Context) {
 	defer redirect(ctx)
 	branchName := ctx.Query("name")
-	if branchName == ctx.Repo.Repository.DefaultBranch {
-		log.Debug("DeleteBranch: Can't delete default branch '%s'", branchName)
-		ctx.Flash.Error(ctx.Tr("repo.branch.default_deletion_failed", branchName))
-		return
-	}
 
-	isProtected, err := ctx.Repo.Repository.IsProtectedBranch(branchName, ctx.User)
-	if err != nil {
-		log.Error("DeleteBranch: %v", err)
-		ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", branchName))
-		return
-	}
+	if err := repo_service.DeleteBranch(ctx.User, ctx.Repo.Repository, ctx.Repo.GitRepo, branchName); err != nil {
+		switch {
+		case git.IsErrBranchNotExist(err):
+			log.Debug("DeleteBranch: Can't delete non existing branch '%s'", branchName)
+			ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", branchName))
+		case errors.Is(err, repo_service.ErrBranchIsDefault):
+			log.Debug("DeleteBranch: Can't delete default branch '%s'", branchName)
+			ctx.Flash.Error(ctx.Tr("repo.branch.default_deletion_failed", branchName))
+		case errors.Is(err, repo_service.ErrBranchIsProtected):
+			log.Debug("DeleteBranch: Can't delete protected branch '%s'", branchName)
+			ctx.Flash.Error(ctx.Tr("repo.branch.protected_deletion_failed", branchName))
+		default:
+			log.Error("DeleteBranch: %v", err)
+			ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", branchName))
+		}
 
-	if isProtected {
-		log.Debug("DeleteBranch: Can't delete protected branch '%s'", branchName)
-		ctx.Flash.Error(ctx.Tr("repo.branch.protected_deletion_failed", branchName))
-		return
-	}
-
-	if !ctx.Repo.GitRepo.IsBranchExist(branchName) {
-		log.Debug("DeleteBranch: Can't delete non existing branch '%s'", branchName)
-		ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", branchName))
-		return
-	}
-
-	if err := deleteBranch(ctx, branchName); err != nil {
-		log.Error("DeleteBranch: %v", err)
-		ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", branchName))
 		return
 	}
 
@@ -168,41 +158,6 @@ func redirect(ctx *context.Context) {
 	})
 }
 
-func deleteBranch(ctx *context.Context, branchName string) error {
-	commit, err := ctx.Repo.GitRepo.GetBranchCommit(branchName)
-	if err != nil {
-		log.Error("GetBranchCommit: %v", err)
-		return err
-	}
-
-	if err := ctx.Repo.GitRepo.DeleteBranch(branchName, git.DeleteBranchOptions{
-		Force: true,
-	}); err != nil {
-		log.Error("DeleteBranch: %v", err)
-		return err
-	}
-
-	// Don't return error below this
-	if err := repo_service.PushUpdate(
-		&repo_module.PushUpdateOptions{
-			RefFullName:  git.BranchPrefix + branchName,
-			OldCommitID:  commit.ID.String(),
-			NewCommitID:  git.EmptySHA,
-			PusherID:     ctx.User.ID,
-			PusherName:   ctx.User.Name,
-			RepoUserName: ctx.Repo.Owner.Name,
-			RepoName:     ctx.Repo.Repository.Name,
-		}); err != nil {
-		log.Error("Update: %v", err)
-	}
-
-	if err := ctx.Repo.Repository.AddDeletedBranch(branchName, commit.ID.String(), ctx.User.ID); err != nil {
-		log.Warn("AddDeletedBranch: %v", err)
-	}
-
-	return nil
-}
-
 // loadBranches loads branches from the repository limited by page & pageSize.
 // NOTE: May write to context on error.
 func loadBranches(ctx *context.Context, skip, limit int) ([]*Branch, int) {

routers/repo/http.go

@@ -447,7 +447,26 @@ func (h *serviceHandler) setHeaderCacheForever() {
 	h.w.Header().Set("Cache-Control", "public, max-age=31536000")
 }
 
+func containsParentDirectorySeparator(v string) bool {
+	if !strings.Contains(v, "..") {
+		return false
+	}
+	for _, ent := range strings.FieldsFunc(v, isSlashRune) {
+		if ent == ".." {
+			return true
+		}
+	}
+	return false
+}
+
+func isSlashRune(r rune) bool { return r == '/' || r == '\\' }
+
 func (h *serviceHandler) sendFile(contentType, file string) {
+	if containsParentDirectorySeparator(file) {
+		log.Error("request file path contains invalid path: %v", file)
+		h.w.WriteHeader(http.StatusBadRequest)
+		return
+	}
 	reqFile := path.Join(h.dir, file)
 
 	fi, err := os.Stat(reqFile)

routers/repo/http_test.go

@@ -0,0 +1,43 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package repo
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestContainsParentDirectorySeparator(t *testing.T) {
+	tests := []struct {
+		v string
+		b bool
+	}{
+		{
+			v: `user2/repo1/info/refs`,
+			b: false,
+		},
+		{
+			v: `user2/repo1/HEAD`,
+			b: false,
+		},
+		{
+			v: `user2/repo1/some.../strange_file...mp3`,
+			b: false,
+		},
+		{
+			v: `user2/repo1/../../custom/conf/app.ini`,
+			b: true,
+		},
+		{
+			v: `user2/repo1/objects/info/..\..\..\..\custom\conf\app.ini`,
+			b: true,
+		},
+	}
+
+	for i := range tests {
+		assert.EqualValues(t, tests[i].b, containsParentDirectorySeparator(tests[i].v))
+	}
+}

routers/repo/pull.go

@@ -9,6 +9,7 @@ package repo
 import (
 	"container/list"
 	"crypto/subtle"
+	"errors"
 	"fmt"
 	"net/http"
 	"path"
@@ -22,7 +23,6 @@ import (
 	"code.gitea.io/gitea/modules/git"
 	"code.gitea.io/gitea/modules/log"
 	"code.gitea.io/gitea/modules/notification"
-	repo_module "code.gitea.io/gitea/modules/repository"
 	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/modules/structs"
 	"code.gitea.io/gitea/modules/upload"
@@ -1186,20 +1186,6 @@ func CleanUpPullRequest(ctx *context.Context) {
 		})
 	}()
 
-	if pr.HeadBranch == pr.HeadRepo.DefaultBranch || !gitRepo.IsBranchExist(pr.HeadBranch) {
-		ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", fullBranchName))
-		return
-	}
-
-	// Check if branch is not protected
-	if protected, err := pr.HeadRepo.IsProtectedBranch(pr.HeadBranch, ctx.User); err != nil || protected {
-		if err != nil {
-			log.Error("HeadRepo.IsProtectedBranch: %v", err)
-		}
-		ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", fullBranchName))
-		return
-	}
-
 	// Check if branch has no new commits
 	headCommitID, err := gitBaseRepo.GetRefCommitID(pr.GetGitRefName())
 	if err != nil {
@@ -1218,27 +1204,21 @@ func CleanUpPullRequest(ctx *context.Context) {
 		return
 	}
 
-	if err := gitRepo.DeleteBranch(pr.HeadBranch, git.DeleteBranchOptions{
-		Force: true,
-	}); err != nil {
-		log.Error("DeleteBranch: %v", err)
-		ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", fullBranchName))
+	if err := repo_service.DeleteBranch(ctx.User, pr.HeadRepo, gitRepo, pr.HeadBranch); err != nil {
+		switch {
+		case git.IsErrBranchNotExist(err):
+			ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", fullBranchName))
+		case errors.Is(err, repo_service.ErrBranchIsDefault):
+			ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", fullBranchName))
+		case errors.Is(err, repo_service.ErrBranchIsProtected):
+			ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", fullBranchName))
+		default:
+			log.Error("DeleteBranch: %v", err)
+			ctx.Flash.Error(ctx.Tr("repo.branch.deletion_failed", fullBranchName))
+		}
 		return
 	}
 
-	if err := repo_service.PushUpdate(
-		&repo_module.PushUpdateOptions{
-			RefFullName:  git.BranchPrefix + pr.HeadBranch,
-			OldCommitID:  branchCommitID,
-			NewCommitID:  git.EmptySHA,
-			PusherID:     ctx.User.ID,
-			PusherName:   ctx.User.Name,
-			RepoUserName: pr.HeadRepo.Owner.Name,
-			RepoName:     pr.HeadRepo.Name,
-		}); err != nil {
-		log.Error("Update: %v", err)
-	}
-
 	if err := models.AddDeletePRBranchComment(ctx.User, pr.BaseRepo, issue.ID, pr.HeadBranch); err != nil {
 		// Do not fail here as branch has already been deleted
 		log.Error("DeleteBranch: %v", err)

routers/repo/setting.go

@@ -164,6 +164,12 @@ func SettingsPost(ctx *context.Context) {
 			}
 		}
 
+		oldUsername := mirror_service.Username(ctx.Repo.Mirror)
+		oldPassword := mirror_service.Password(ctx.Repo.Mirror)
+		if form.MirrorPassword == "" && form.MirrorUsername == oldUsername {
+			form.MirrorPassword = oldPassword
+		}
+
 		address, err := auth.ParseRemoteAddr(form.MirrorAddress, form.MirrorUsername, form.MirrorPassword)
 		if err == nil {
 			err = migrations.IsMigrateURLAllowed(address, ctx.User)
@@ -539,6 +545,11 @@ func SettingsPost(ctx *context.Context) {
 			return
 		}
 
+		// Close the gitrepository before doing this.
+		if ctx.Repo.GitRepo != nil {
+			ctx.Repo.GitRepo.Close()
+		}
+
 		if err := repo_service.DeleteRepository(ctx.User, ctx.Repo.Repository); err != nil {
 			ctx.ServerError("DeleteRepository", err)
 			return

routers/routes/goget.go

@@ -0,0 +1,86 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package routes
+
+import (
+	"net/http"
+	"net/url"
+	"path"
+	"strings"
+
+	"code.gitea.io/gitea/models"
+	"code.gitea.io/gitea/modules/context"
+	"code.gitea.io/gitea/modules/setting"
+	"code.gitea.io/gitea/modules/util"
+	"github.com/unknwon/com"
+)
+
+func goGet(ctx *context.Context) {
+	if ctx.Req.Method != "GET" || ctx.Query("go-get") != "1" || len(ctx.Req.URL.Query()) > 1 {
+		return
+	}
+
+	parts := strings.SplitN(ctx.Req.URL.EscapedPath(), "/", 4)
+
+	if len(parts) < 3 {
+		return
+	}
+
+	ownerName := parts[1]
+	repoName := parts[2]
+
+	// Quick responses appropriate go-get meta with status 200
+	// regardless of if user have access to the repository,
+	// or the repository does not exist at all.
+	// This is particular a workaround for "go get" command which does not respect
+	// .netrc file.
+
+	trimmedRepoName := strings.TrimSuffix(repoName, ".git")
+
+	if ownerName == "" || trimmedRepoName == "" {
+		_, _ = ctx.Write([]byte(`<!doctype html>
+<html>
+	<body>
+		invalid import path
+	</body>
+</html>
+`))
+		ctx.Status(400)
+		return
+	}
+	branchName := setting.Repository.DefaultBranch
+
+	repo, err := models.GetRepositoryByOwnerAndName(ownerName, repoName)
+	if err == nil && len(repo.DefaultBranch) > 0 {
+		branchName = repo.DefaultBranch
+	}
+	prefix := setting.AppURL + path.Join(url.PathEscape(ownerName), url.PathEscape(repoName), "src", "branch", util.PathEscapeSegments(branchName))
+
+	appURL, _ := url.Parse(setting.AppURL)
+
+	insecure := ""
+	if appURL.Scheme == string(setting.HTTP) {
+		insecure = "--insecure "
+	}
+	ctx.Header().Set("Content-Type", "text/html")
+	ctx.Status(http.StatusOK)
+	_, _ = ctx.Write([]byte(com.Expand(`<!doctype html>
+<html>
+	<head>
+		<meta name="go-import" content="{GoGetImport} git {CloneLink}">
+		<meta name="go-source" content="{GoGetImport} _ {GoDocDirectory} {GoDocFile}">
+	</head>
+	<body>
+		go get {Insecure}{GoGetImport}
+	</body>
+</html>
+`, map[string]string{
+		"GoGetImport":    context.ComposeGoGetImport(ownerName, trimmedRepoName),
+		"CloneLink":      models.ComposeHTTPSCloneURL(ownerName, repoName),
+		"GoDocDirectory": prefix + "{/dir}",
+		"GoDocFile":      prefix + "{/dir}/{file}#L{line}",
+		"Insecure":       insecure,
+	})))
+}

routers/routes/install.go

@@ -89,6 +89,7 @@ func InstallRoutes() *web.Route {
 		Gclifetime:     setting.SessionConfig.Gclifetime,
 		Maxlifetime:    setting.SessionConfig.Maxlifetime,
 		Secure:         setting.SessionConfig.Secure,
+		SameSite:       setting.SessionConfig.SameSite,
 		Domain:         setting.SessionConfig.Domain,
 	}))
 
@@ -110,7 +111,7 @@ func InstallRoutes() *web.Route {
 	r.Get("/", routers.Install)
 	r.Post("/", web.Bind(forms.InstallForm{}), routers.InstallPost)
 	r.NotFound(func(w http.ResponseWriter, req *http.Request) {
-		http.Redirect(w, req, setting.AppURL, 302)
+		http.Redirect(w, req, setting.AppURL, http.StatusFound)
 	})
 	return r
 }

routers/routes/web.go

@@ -8,7 +8,6 @@ import (
 	"encoding/gob"
 	"fmt"
 	"net/http"
-	"net/url"
 	"os"
 	"path"
 	"strings"
@@ -24,7 +23,6 @@ import (
 	"code.gitea.io/gitea/modules/setting"
 	"code.gitea.io/gitea/modules/storage"
 	"code.gitea.io/gitea/modules/templates"
-	"code.gitea.io/gitea/modules/util"
 	"code.gitea.io/gitea/modules/validation"
 	"code.gitea.io/gitea/modules/web"
 	"code.gitea.io/gitea/routers"
@@ -51,7 +49,6 @@ import (
 	"github.com/go-chi/cors"
 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/tstranex/u2f"
-	"github.com/unknwon/com"
 )
 
 const (
@@ -138,6 +135,7 @@ func WebRoutes() *web.Route {
 		Gclifetime:     setting.SessionConfig.Gclifetime,
 		Maxlifetime:    setting.SessionConfig.Maxlifetime,
 		Secure:         setting.SessionConfig.Secure,
+		SameSite:       setting.SessionConfig.SameSite,
 		Domain:         setting.SessionConfig.Domain,
 	}))
 
@@ -190,6 +188,7 @@ func WebRoutes() *web.Route {
 		ctx.Data["UnitPullsGlobalDisabled"] = models.UnitTypePullRequests.UnitGlobalDisabled()
 		ctx.Data["UnitProjectsGlobalDisabled"] = models.UnitTypeProjects.UnitGlobalDisabled()
 	})
+	r.Use(goGet)
 
 	// for health check
 	r.Head("/", func(w http.ResponseWriter, req *http.Request) {
@@ -229,67 +228,6 @@ func WebRoutes() *web.Route {
 	return r
 }
 
-func goGet(ctx *context.Context) {
-	if ctx.Query("go-get") != "1" {
-		return
-	}
-
-	// Quick responses appropriate go-get meta with status 200
-	// regardless of if user have access to the repository,
-	// or the repository does not exist at all.
-	// This is particular a workaround for "go get" command which does not respect
-	// .netrc file.
-
-	ownerName := ctx.Params(":username")
-	repoName := ctx.Params(":reponame")
-	trimmedRepoName := strings.TrimSuffix(repoName, ".git")
-
-	if ownerName == "" || trimmedRepoName == "" {
-		_, _ = ctx.Write([]byte(`<!doctype html>
-<html>
-	<body>
-		invalid import path
-	</body>
-</html>
-`))
-		ctx.Status(400)
-		return
-	}
-	branchName := setting.Repository.DefaultBranch
-
-	repo, err := models.GetRepositoryByOwnerAndName(ownerName, repoName)
-	if err == nil && len(repo.DefaultBranch) > 0 {
-		branchName = repo.DefaultBranch
-	}
-	prefix := setting.AppURL + path.Join(url.PathEscape(ownerName), url.PathEscape(repoName), "src", "branch", util.PathEscapeSegments(branchName))
-
-	appURL, _ := url.Parse(setting.AppURL)
-
-	insecure := ""
-	if appURL.Scheme == string(setting.HTTP) {
-		insecure = "--insecure "
-	}
-	ctx.Header().Set("Content-Type", "text/html")
-	ctx.Status(http.StatusOK)
-	_, _ = ctx.Write([]byte(com.Expand(`<!doctype html>
-<html>
-	<head>
-		<meta name="go-import" content="{GoGetImport} git {CloneLink}">
-		<meta name="go-source" content="{GoGetImport} _ {GoDocDirectory} {GoDocFile}">
-	</head>
-	<body>
-		go get {Insecure}{GoGetImport}
-	</body>
-</html>
-`, map[string]string{
-		"GoGetImport":    context.ComposeGoGetImport(ownerName, trimmedRepoName),
-		"CloneLink":      models.ComposeHTTPSCloneURL(ownerName, repoName),
-		"GoDocDirectory": prefix + "{/dir}",
-		"GoDocFile":      prefix + "{/dir}/{file}#L{line}",
-		"Insecure":       insecure,
-	})))
-}
-
 // RegisterRoutes register routes
 func RegisterRoutes(m *web.Route) {
 	reqSignIn := context.Toggle(&context.ToggleOptions{SignInRequired: true})
@@ -1091,7 +1029,7 @@ func RegisterRoutes(m *web.Route) {
 	m.Group("/{username}", func() {
 		m.Group("/{reponame}", func() {
 			m.Get("", repo.SetEditorconfigIfExists, repo.Home)
-		}, goGet, ignSignIn, context.RepoAssignment, context.RepoRef(), context.UnitTypes())
+		}, ignSignIn, context.RepoAssignment, context.RepoRef(), context.UnitTypes())
 
 		m.Group("/{reponame}", func() {
 			m.Group("/info/lfs", func() {

routers/user/setting/profile.go

@@ -67,8 +67,13 @@ func HandleUsernameChange(ctx *context.Context, user *models.User, newName strin
 			}
 			return err
 		}
-		log.Trace("User name changed: %s -> %s", user.Name, newName)
+	} else {
+		if err := models.UpdateRepositoryOwnerNames(user.ID, newName); err != nil {
+			ctx.ServerError("UpdateRepository", err)
+			return err
+		}
 	}
+	log.Trace("User name changed: %s -> %s", user.Name, newName)
 	return nil
 }
 
@@ -84,6 +89,7 @@ func ProfilePost(ctx *context.Context) {
 	}
 
 	if len(form.Name) != 0 && ctx.User.Name != form.Name {
+		log.Debug("Changing name for %s to %s", ctx.User.Name, form.Name)
 		if err := HandleUsernameChange(ctx, ctx.User, form.Name); err != nil {
 			ctx.Redirect(setting.AppSubURL + "/user/settings")
 			return

services/mailer/mail_comment.go

@@ -20,12 +20,16 @@ func mailParticipantsComment(c *models.Comment, opType models.ActionType, issue
 	for i, u := range mentions {
 		mentionedIDs[i] = u.ID
 	}
+	content := c.Content
+	if c.Type == models.CommentTypePullPush {
+		content = ""
+	}
 	if err = mailIssueCommentToParticipants(
 		&mailCommentContext{
 			Issue:      issue,
 			Doer:       c.Poster,
 			ActionType: opType,
-			Content:    c.Content,
+			Content:    content,
 			Comment:    c,
 		}, mentionedIDs); err != nil {
 		log.Error("mailIssueCommentToParticipants: %v", err)

services/mailer/mail_issue.go

@@ -158,12 +158,18 @@ func mailParticipants(issue *models.Issue, doer *models.User, opType models.Acti
 	for i, u := range mentions {
 		mentionedIDs[i] = u.ID
 	}
+	content := issue.Content
+	if opType == models.ActionCloseIssue || opType == models.ActionClosePullRequest ||
+		opType == models.ActionReopenIssue || opType == models.ActionReopenPullRequest ||
+		opType == models.ActionMergePullRequest {
+		content = ""
+	}
 	if err = mailIssueCommentToParticipants(
 		&mailCommentContext{
 			Issue:      issue,
 			Doer:       doer,
 			ActionType: opType,
-			Content:    issue.Content,
+			Content:    content,
 			Comment:    nil,
 		}, mentionedIDs); err != nil {
 		log.Error("mailIssueCommentToParticipants: %v", err)

services/repository/branch.go

@@ -0,0 +1,72 @@
+// Copyright 2021 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package repository
+
+import (
+	"errors"
+
+	"code.gitea.io/gitea/models"
+	"code.gitea.io/gitea/modules/git"
+	"code.gitea.io/gitea/modules/log"
+	repo_module "code.gitea.io/gitea/modules/repository"
+	pull_service "code.gitea.io/gitea/services/pull"
+)
+
+// enmuerates all branch related errors
+var (
+	ErrBranchIsDefault   = errors.New("branch is default")
+	ErrBranchIsProtected = errors.New("branch is protected")
+)
+
+// DeleteBranch delete branch
+func DeleteBranch(doer *models.User, repo *models.Repository, gitRepo *git.Repository, branchName string) error {
+	if branchName == repo.DefaultBranch {
+		return ErrBranchIsDefault
+	}
+
+	isProtected, err := repo.IsProtectedBranch(branchName, doer)
+	if err != nil {
+		return err
+	}
+
+	if isProtected {
+		return ErrBranchIsProtected
+	}
+
+	commit, err := gitRepo.GetBranchCommit(branchName)
+	if err != nil {
+		return err
+	}
+
+	if err := gitRepo.DeleteBranch(branchName, git.DeleteBranchOptions{
+		Force: true,
+	}); err != nil {
+		return err
+	}
+
+	if err := pull_service.CloseBranchPulls(doer, repo.ID, branchName); err != nil {
+		return err
+	}
+
+	// Don't return error below this
+	if err := PushUpdate(
+		&repo_module.PushUpdateOptions{
+			RefFullName:  git.BranchPrefix + branchName,
+			OldCommitID:  commit.ID.String(),
+			NewCommitID:  git.EmptySHA,
+			PusherID:     doer.ID,
+			PusherName:   doer.Name,
+			RepoUserName: repo.OwnerName,
+			RepoName:     repo.Name,
+		}); err != nil {
+		log.Error("Update: %v", err)
+	}
+
+	if err := repo.AddDeletedBranch(branchName, commit.ID.String(), doer.ID); err != nil {
+		log.Warn("AddDeletedBranch: %v", err)
+	}
+
+	return nil
+}

services/repository/push.go

@@ -193,16 +193,17 @@ func pushUpdates(optsList []*repo_module.PushUpdateOptions) error {
 				}
 
 				commits = repo_module.ListToPushCommits(l)
+
+				if err := repofiles.UpdateIssuesCommit(pusher, repo, commits.Commits, refName); err != nil {
+					log.Error("updateIssuesCommit: %v", err)
+				}
+
 				if len(commits.Commits) > setting.UI.FeedMaxCommitNum {
 					commits.Commits = commits.Commits[:setting.UI.FeedMaxCommitNum]
 				}
 				commits.CompareURL = repo.ComposeCompareURL(opts.OldCommitID, opts.NewCommitID)
 				notification.NotifyPushCommits(pusher, repo, opts, commits)
 
-				if err := repofiles.UpdateIssuesCommit(pusher, repo, commits.Commits, refName); err != nil {
-					log.Error("updateIssuesCommit: %v", err)
-				}
-
 				if err = models.RemoveDeletedBranch(repo.ID, branch); err != nil {
 					log.Error("models.RemoveDeletedBranch %s/%s failed: %v", repo.ID, branch, err)
 				}

snap/snapcraft.yaml

@@ -25,14 +25,14 @@ environment:
 apps:
   gitea:
     command: gitea
-    plugs: [network, network-bind]
+    plugs: [network, network-bind, removable-media]
   web:
     command: gitea web
     daemon: simple
-    plugs: [network, network-bind]
+    plugs: [network, network-bind, removable-media]
   dump:
     command: gitea dump
-    plugs: [home]
+    plugs: [home, removable-media]
   version:
     command: gitea --version
   sqlite:

templates/admin/auth/edit.tmpl

@@ -188,6 +188,10 @@
 						<label for="pam_service_name">{{.i18n.Tr "admin.auths.pam_service_name"}}</label>
 						<input id="pam_service_name" name="pam_service_name" value="{{$cfg.ServiceName}}" required>
 					</div>
+					<div class="field">
+						<label for="pam_email_domain">{{.i18n.Tr "admin.auths.pam_email_domain"}}</label>
+						<input id="pam_email_domain" name="pam_email_domain" value="{{$cfg.EmailDomain}}">
+					</div>
 				{{end}}
 
 				<!-- OAuth2 -->

templates/admin/auth/new.tmpl

@@ -38,6 +38,8 @@
 				<div class="pam required field {{if not (eq .type 4)}}hide{{end}}">
 					<label for="pam_service_name">{{.i18n.Tr "admin.auths.pam_service_name"}}</label>
 					<input id="pam_service_name" name="pam_service_name" value="{{.pam_service_name}}" />
+					<label for="pam_email_domain">{{.i18n.Tr "admin.auths.pam_email_domain"}}</label>
+					<input id="pam_email_domain" name="pam_email_domain" value="{{.pam_email_domain}}">
 				</div>
 
 				<!-- OAuth2 -->

templates/repo/branch_dropdown.tmpl

@@ -19,7 +19,7 @@
 		<div class="menu transition" :class="{visible: menuVisible}" v-if="menuVisible" v-cloak>
 			<div class="ui icon search input">
 				<i class="icon df ac jc m-0">{{svg "octicon-filter" 16}}</i>
-				<input name="search" ref="searchField" v-model="searchTerm" @keydown="keydown($event)" placeholder="{{.i18n.Tr "repo.filter_branch_and_tag"}}...">
+				<input name="search" ref="searchField" autocomplete="off" v-model="searchTerm" @keydown="keydown($event)" placeholder="{{.i18n.Tr "repo.filter_branch_and_tag"}}...">
 			</div>
 			<div class="header branch-tag-choice">
 				<div class="ui grid">