Compare commits

...

22 Commits

Author SHA1 Message Date
6543
6dfa92bb1c Changelog v1.13.6 (#15129) 2021-03-23 15:44:50 -04:00
6543
151bedab52 Fix bug on avatar middleware (#15125)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-03-23 18:45:06 +00:00
zeripath
6198403fbc Fix another clusterfuzz identified issue (#15096) (#15114)
Backport #15096

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-03-22 16:27:21 -04:00
a1012112796
a6290f603f fix #15104 (#15106)
Signed-off-by: a1012112796 <1012112796@qq.com>
2021-03-22 15:15:44 -04:00
silverwind
2f09e5775f Fix markdown rendering in milestone content (#15056) (#15092)
- Add missing markdown class for rendered markdown.
- Increase font size of milestone name in list.

Fixes: https://github.com/go-gitea/gitea/issues/15046
2021-03-21 18:03:52 +01:00
zeripath
b0819efaea Place wrapper around comment as diff to catch panics (#15085) (#15086)
* Place wrapper around comment as diff to prevent panics

* propagate the panic up

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-21 16:16:07 +01:00
6543
d7a3bcdd70 Changelog v1.13.5 (#15084) 2021-03-21 15:05:21 +01:00
zeripath
7a85e228d8 Update to goldmark 1.3.3 (#15059) (#15061)
Backport #15059

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-20 10:31:28 +00:00
6543
a461d90415 Fix bug when upload on web (#15042) (#15055)
* Fix bug when upload on web

* move into own function

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
2021-03-20 09:37:53 +08:00
6543
70e4134130 Delete Labels & IssueLabels on Repo Delete too (#15039) (#15051)
* Doctor: find IssueLabels without existing label

* Repo Delete: delete labels & issue_labels too
2021-03-19 22:13:39 +01:00
zeripath
909f2be99d Fix postgres ID sequences broken by recreate-table (#15015) (#15029)
Backport #15015

Unfortunately there is a subtle problem with recreatetable on postgres which
leads to the sequences not being renamed and not being left at 0.

Fix #14725

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-19 04:23:58 +01:00
6543
645c0d8abd another clusterfuzz spotted issue (#15032) (#15034)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-03-19 00:21:33 +02:00
zeripath
8c461eb261 Fix several render issues (#14986) (#15013)
Backport #14986

* Fix an issue with panics related to attributes
* Wrap goldmark render in a recovery function
* Reduce memory use in render emoji
* Use a pipe for rendering goldmark - still needs more work and a limiter

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-03-17 10:58:58 +02:00
Norwin
fff66eb016 API: fix set milestone on PR creation (#14981) (#15001)
* API: fix set milestone on PR creation

pr creation via API failed with 404, because we searched
for milestoneID 0, due to uninitialized var usage D:

* add tests

Co-authored-by: 6543 <6543@obermui.de>

Co-authored-by: 6543 <6543@obermui.de>
2021-03-15 11:01:04 -04:00
zeripath
c965ed6529 Make sure sibling images get a link too (#14979) (#14995)
Backport #14979

Due a problem with the ast.Walker in the our transformer in goldmark
an image with a sibling image will not be transformed to gain a parent
link. This PR fixes this.

Fix #12925

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-15 12:34:56 +08:00
zeripath
71a2adbf10 Fix Anchor jumping with escaped query components (#14969) (#14977)
Backport #14969

Fix #14968

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-13 09:54:53 +00:00
Norwin
3231b70043 check if original author is set (#14972)
Co-authored-by: 6543 <6543@obermui.de>
2021-03-13 11:05:56 +08:00
Norwin
e3c44923d7 fix release mail html template (#14976)
was missing an </a>
2021-03-12 20:39:05 +00:00
zeripath
3e7dccdf47 Fix excluding more than two labels on issues list (#14962) (#14973)
Backport #14962

Fix #14840

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Norwin Roosen <git@nroo.de>
Co-authored-by: jaqra <48099350+jaqra@users.noreply.github.com>

Co-authored-by: Norwin Roosen <git@nroo.de>
Co-authored-by: jaqra <48099350+jaqra@users.noreply.github.com>
2021-03-12 18:12:14 +01:00
6543
33c2c49627 Prevent panic when editing forked repos by API (#14960) (#14963)
When editing forked repos using the API the BaseRepository needs to loaded
in order to check its visibility otherwise there will be NPE panic.

Fix #14956

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-03-12 08:54:18 +08:00
fnetX (aka fralix)
05ac72cf33 Add "captcha" to list of reserved usernames (#14930)
Signed-off-by: Otto Richter <git@fralix.ovh>
2021-03-08 17:50:13 +01:00
zeripath
906ecfd173 Re-enable import local paths after reversion from #13610 (#14925) (#14927)
Backport #14925

PR #13610 unfortunately disabled importing repositories from local paths.
This PR restores this functionality.

Fix #14700

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-08 14:50:57 +01:00
50 changed files with 1356 additions and 194 deletions

View File

@@ -4,6 +4,37 @@ This changelog goes through all the changes that have been made in each release
without substantial changes to our git log; to see the highlights of what has without substantial changes to our git log; to see the highlights of what has
been added to each release, please refer to the [blog](https://blog.gitea.io). been added to each release, please refer to the [blog](https://blog.gitea.io).
## [1.13.6](https://github.com/go-gitea/gitea/releases/tag/v1.13.6) - 2021-03-23
* SECURITY
* Fix bug on avatar middleware (#15124) (#15125)
* Fix another clusterfuzz identified issue (#15096) (#15114)
* API
* Fix nil exeption for get pull reviews API #15104 (#15106)
* BUGFIXES
* Fix markdown rendering in milestone content (#15056) (#15092)
## [1.13.5](https://github.com/go-gitea/gitea/releases/tag/v1.13.5) - 2021-03-21
* SECURITY
* Update to goldmark 1.3.3 (#15059) (#15061)
* API
* Fix set milestone on PR creation (#14981) (#15001)
* Prevent panic when editing forked repos by API (#14960) (#14963)
* BUGFIXES
* Fix bug when upload on web (#15042) (#15055)
* Delete Labels & IssueLabels on Repo Delete too (#15039) (#15051)
* another clusterfuzz spotted issue (#15032) (#15034)
* Fix postgres ID sequences broken by recreate-table (#15015) (#15029)
* Fix several render issues (#14986) (#15013)
* Make sure sibling images get a link too (#14979) (#14995)
* Fix Anchor jumping with escaped query components (#14969) (#14977)
* fix release mail html template (#14976)
* Fix excluding more than two labels on issues list (#14962) (#14973)
* don't mark each comment poster as OP (#14971) (#14972)
* Add "captcha" to list of reserved usernames (#14930)
* Re-enable import local paths after reversion from #13610 (#14925) (#14927)
## [1.13.4](https://github.com/go-gitea/gitea/releases/tag/v1.13.4) - 2021-03-07 ## [1.13.4](https://github.com/go-gitea/gitea/releases/tag/v1.13.4) - 2021-03-07
* SECURITY * SECURITY

View File

@@ -606,6 +606,22 @@ func runDoctorCheckDBConsistency(ctx *cli.Context) ([]string, error) {
} }
} }
// find IssueLabels without existing label
count, err = models.CountOrphanedIssueLabels()
if err != nil {
return nil, err
}
if count > 0 {
if ctx.Bool("fix") {
if err = models.DeleteOrphanedIssueLabels(); err != nil {
return nil, err
}
results = append(results, fmt.Sprintf("%d issue_labels without existing label deleted", count))
} else {
results = append(results, fmt.Sprintf("%d issue_labels without existing label", count))
}
}
//find issues without existing repository //find issues without existing repository
count, err = models.CountOrphanedIssues() count, err = models.CountOrphanedIssues()
if err != nil { if err != nil {
@@ -670,6 +686,23 @@ func runDoctorCheckDBConsistency(ctx *cli.Context) ([]string, error) {
} }
} }
if setting.Database.UsePostgreSQL {
count, err = models.CountBadSequences()
if err != nil {
return nil, err
}
if count > 0 {
if ctx.Bool("fix") {
err := models.FixBadSequences()
if err != nil {
return nil, err
}
results = append(results, fmt.Sprintf("%d sequences updated", count))
} else {
results = append(results, fmt.Sprintf("%d sequences with incorrect values", count))
}
}
}
//ToDo: function to recalc all counters //ToDo: function to recalc all counters
return results, nil return results, nil

2
go.mod
View File

@@ -99,7 +99,7 @@ require (
github.com/urfave/cli v1.20.0 github.com/urfave/cli v1.20.0
github.com/xanzy/go-gitlab v0.37.0 github.com/xanzy/go-gitlab v0.37.0
github.com/yohcop/openid-go v1.0.0 github.com/yohcop/openid-go v1.0.0
github.com/yuin/goldmark v1.2.1 github.com/yuin/goldmark v1.3.3
github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691 github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691
github.com/yuin/goldmark-meta v0.0.0-20191126180153-f0638e958b60 github.com/yuin/goldmark-meta v0.0.0-20191126180153-f0638e958b60
go.jolheiser.com/hcaptcha v0.0.4 go.jolheiser.com/hcaptcha v0.0.4

2
go.sum
View File

@@ -887,6 +887,8 @@ github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1 h1:ruQGxdhGHe7FWOJPT0mKs5+pD2Xs1Bm/kdGlHO04FmM= github.com/yuin/goldmark v1.2.1 h1:ruQGxdhGHe7FWOJPT0mKs5+pD2Xs1Bm/kdGlHO04FmM=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.3.3 h1:37BdQwPx8VOSic8eDSWee6QL9mRpZRm9VJp/QugNrW0=
github.com/yuin/goldmark v1.3.3/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691 h1:VWSxtAiQNh3zgHJpdpkpVYjTPqRE3P6UZCOPa1nRDio= github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691 h1:VWSxtAiQNh3zgHJpdpkpVYjTPqRE3P6UZCOPa1nRDio=
github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691/go.mod h1:YLF3kDffRfUH/bTxOxHhV6lxwIB3Vfj91rEwNMS9MXo= github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691/go.mod h1:YLF3kDffRfUH/bTxOxHhV6lxwIB3Vfj91rEwNMS9MXo=
github.com/yuin/goldmark-meta v0.0.0-20191126180153-f0638e958b60 h1:gZucqLjL1eDzVWrXj4uiWeMbAopJlBR2mKQAsTGdPwo= github.com/yuin/goldmark-meta v0.0.0-20191126180153-f0638e958b60 h1:gZucqLjL1eDzVWrXj4uiWeMbAopJlBR2mKQAsTGdPwo=

View File

@@ -74,8 +74,79 @@ func TestAPICreatePullSuccess(t *testing.T) {
Base: "master", Base: "master",
Title: "create a failure pr", Title: "create a failure pr",
}) })
session.MakeRequest(t, req, 201) session.MakeRequest(t, req, 201)
session.MakeRequest(t, req, http.StatusUnprocessableEntity) // second request should fail
}
func TestAPICreatePullWithFieldsSuccess(t *testing.T) {
defer prepareTestEnv(t)()
// repo10 have code, pulls units.
repo10 := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 10}).(*models.Repository)
owner10 := models.AssertExistsAndLoadBean(t, &models.User{ID: repo10.OwnerID}).(*models.User)
// repo11 only have code unit but should still create pulls
repo11 := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 11}).(*models.Repository)
owner11 := models.AssertExistsAndLoadBean(t, &models.User{ID: repo11.OwnerID}).(*models.User)
session := loginUser(t, owner11.Name)
token := getTokenForLoggedInUser(t, session)
opts := &api.CreatePullRequestOption{
Head: fmt.Sprintf("%s:master", owner11.Name),
Base: "master",
Title: "create a failure pr",
Body: "foobaaar",
Milestone: 5,
Assignees: []string{owner10.Name},
Labels: []int64{5},
}
req := NewRequestWithJSON(t, http.MethodPost, fmt.Sprintf("/api/v1/repos/%s/%s/pulls?token=%s", owner10.Name, repo10.Name, token), opts)
res := session.MakeRequest(t, req, 201)
pull := new(api.PullRequest)
DecodeJSON(t, res, pull)
assert.NotNil(t, pull.Milestone)
assert.EqualValues(t, opts.Milestone, pull.Milestone.ID)
if assert.Len(t, pull.Assignees, 1) {
assert.EqualValues(t, opts.Assignees[0], owner10.Name)
}
assert.NotNil(t, pull.Labels)
assert.EqualValues(t, opts.Labels[0], pull.Labels[0].ID)
}
func TestAPICreatePullWithFieldsFailure(t *testing.T) {
defer prepareTestEnv(t)()
// repo10 have code, pulls units.
repo10 := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 10}).(*models.Repository)
owner10 := models.AssertExistsAndLoadBean(t, &models.User{ID: repo10.OwnerID}).(*models.User)
// repo11 only have code unit but should still create pulls
repo11 := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 11}).(*models.Repository)
owner11 := models.AssertExistsAndLoadBean(t, &models.User{ID: repo11.OwnerID}).(*models.User)
session := loginUser(t, owner11.Name)
token := getTokenForLoggedInUser(t, session)
opts := &api.CreatePullRequestOption{
Head: fmt.Sprintf("%s:master", owner11.Name),
Base: "master",
}
req := NewRequestWithJSON(t, http.MethodPost, fmt.Sprintf("/api/v1/repos/%s/%s/pulls?token=%s", owner10.Name, repo10.Name, token), opts)
session.MakeRequest(t, req, http.StatusUnprocessableEntity)
opts.Title = "is required"
opts.Milestone = 666
session.MakeRequest(t, req, http.StatusUnprocessableEntity)
opts.Milestone = 5
opts.Assignees = []string{"qweruqweroiuyqweoiruywqer"}
session.MakeRequest(t, req, http.StatusUnprocessableEntity)
opts.Assignees = []string{owner10.LoginName}
opts.Labels = []int64{55555}
session.MakeRequest(t, req, http.StatusUnprocessableEntity)
opts.Labels = []int64{5}
} }
func TestAPIEditPull(t *testing.T) { func TestAPIEditPull(t *testing.T) {

View File

@@ -5,10 +5,13 @@
package models package models
import ( import (
"fmt"
"reflect" "reflect"
"regexp"
"strings" "strings"
"testing" "testing"
"code.gitea.io/gitea/modules/setting"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"xorm.io/builder" "xorm.io/builder"
) )
@@ -221,6 +224,24 @@ func DeleteOrphanedLabels() error {
return nil return nil
} }
// CountOrphanedIssueLabels return count of IssueLabels witch have no label behind anymore
func CountOrphanedIssueLabels() (int64, error) {
return x.Table("issue_label").
Join("LEFT", "label", "issue_label.label_id = label.id").
Where(builder.IsNull{"label.id"}).Count()
}
// DeleteOrphanedIssueLabels delete IssueLabels witch have no label behind anymore
func DeleteOrphanedIssueLabels() error {
_, err := x.In("id", builder.Select("issue_label.id").From("issue_label").
Join("LEFT", "label", "issue_label.label_id = label.id").
Where(builder.IsNull{"label.id"})).
Delete(IssueLabel{})
return err
}
// CountOrphanedIssues count issues without a repo // CountOrphanedIssues count issues without a repo
func CountOrphanedIssues() (int64, error) { func CountOrphanedIssues() (int64, error) {
return x.Table("issue"). return x.Table("issue").
@@ -295,3 +316,61 @@ func FixNullArchivedRepository() (int64, error) {
IsArchived: false, IsArchived: false,
}) })
} }
// CountBadSequences looks for broken sequences from recreate-table mistakes
func CountBadSequences() (int64, error) {
if !setting.Database.UsePostgreSQL {
return 0, nil
}
sess := x.NewSession()
defer sess.Close()
var sequences []string
schema := sess.Engine().Dialect().URI().Schema
sess.Engine().SetSchema("")
if err := sess.Table("information_schema.sequences").Cols("sequence_name").Where("sequence_name LIKE 'tmp_recreate__%_id_seq%' AND sequence_catalog = ?", setting.Database.Name).Find(&sequences); err != nil {
return 0, err
}
sess.Engine().SetSchema(schema)
return int64(len(sequences)), nil
}
// FixBadSequences fixes for broken sequences from recreate-table mistakes
func FixBadSequences() error {
if !setting.Database.UsePostgreSQL {
return nil
}
sess := x.NewSession()
defer sess.Close()
if err := sess.Begin(); err != nil {
return err
}
var sequences []string
schema := sess.Engine().Dialect().URI().Schema
sess.Engine().SetSchema("")
if err := sess.Table("information_schema.sequences").Cols("sequence_name").Where("sequence_name LIKE 'tmp_recreate__%_id_seq%' AND sequence_catalog = ?", setting.Database.Name).Find(&sequences); err != nil {
return err
}
sess.Engine().SetSchema(schema)
sequenceRegexp := regexp.MustCompile(`tmp_recreate__(\w+)_id_seq.*`)
for _, sequence := range sequences {
tableName := sequenceRegexp.FindStringSubmatch(sequence)[1]
newSequenceName := tableName + "_id_seq"
if _, err := sess.Exec(fmt.Sprintf("ALTER SEQUENCE `%s` RENAME TO `%s`", sequence, newSequenceName)); err != nil {
return err
}
if _, err := sess.Exec(fmt.Sprintf("SELECT setval('%s', COALESCE((SELECT MAX(id)+1 FROM `%s`), 1), false)", newSequenceName, tableName)); err != nil {
return err
}
}
return sess.Commit()
}

View File

@@ -33,3 +33,11 @@
num_issues: 1 num_issues: 1
num_closed_issues: 0 num_closed_issues: 0
-
id: 5
repo_id: 10
org_id: 0
name: pull-test-label
color: '#000000'
num_issues: 0
num_closed_issues: 0

View File

@@ -29,3 +29,11 @@
content: content random content: content random
is_closed: false is_closed: false
num_issues: 0 num_issues: 0
-
id: 5
repo_id: 10
name: milestone of repo 10
content: for testing with PRs
is_closed: false
num_issues: 0

View File

@@ -146,6 +146,7 @@
num_closed_issues: 0 num_closed_issues: 0
num_pulls: 1 num_pulls: 1
num_closed_pulls: 0 num_closed_pulls: 0
num_milestones: 1
is_mirror: false is_mirror: false
num_forks: 1 num_forks: 1
status: 0 status: 0

View File

@@ -764,3 +764,15 @@ func DeleteIssueLabel(issue *Issue, label *Label, doer *User) (err error) {
return sess.Commit() return sess.Commit()
} }
func deleteLabelsByRepoID(sess Engine, repoID int64) error {
deleteCond := builder.Select("id").From("label").Where(builder.Eq{"label.repo_id": repoID})
if _, err := sess.In("label_id", deleteCond).
Delete(&IssueLabel{}); err != nil {
return err
}
_, err := sess.Delete(&Label{RepoID: repoID})
return err
}

View File

@@ -516,6 +516,31 @@ func recreateTable(sess *xorm.Session, bean interface{}) error {
return err return err
} }
case setting.Database.UsePostgreSQL: case setting.Database.UsePostgreSQL:
var originalSequences []string
type sequenceData struct {
LastValue int `xorm:"'last_value'"`
IsCalled bool `xorm:"'is_called'"`
}
sequenceMap := map[string]sequenceData{}
schema := sess.Engine().Dialect().URI().Schema
sess.Engine().SetSchema("")
if err := sess.Table("information_schema.sequences").Cols("sequence_name").Where("sequence_name LIKE ? || '_%' AND sequence_catalog = ?", tableName, setting.Database.Name).Find(&originalSequences); err != nil {
log.Error("Unable to rename %s to %s. Error: %v", tempTableName, tableName, err)
return err
}
sess.Engine().SetSchema(schema)
for _, sequence := range originalSequences {
sequenceData := sequenceData{}
if _, err := sess.Table(sequence).Cols("last_value", "is_called").Get(&sequenceData); err != nil {
log.Error("Unable to get last_value and is_called from %s. Error: %v", sequence, err)
return err
}
sequenceMap[sequence] = sequenceData
}
// CASCADE causes postgres to drop all the constraints on the old table // CASCADE causes postgres to drop all the constraints on the old table
if _, err := sess.Exec(fmt.Sprintf("DROP TABLE `%s` CASCADE", tableName)); err != nil { if _, err := sess.Exec(fmt.Sprintf("DROP TABLE `%s` CASCADE", tableName)); err != nil {
log.Error("Unable to drop old table %s. Error: %v", tableName, err) log.Error("Unable to drop old table %s. Error: %v", tableName, err)
@@ -529,7 +554,6 @@ func recreateTable(sess *xorm.Session, bean interface{}) error {
} }
var indices []string var indices []string
schema := sess.Engine().Dialect().URI().Schema
sess.Engine().SetSchema("") sess.Engine().SetSchema("")
if err := sess.Table("pg_indexes").Cols("indexname").Where("tablename = ? ", tableName).Find(&indices); err != nil { if err := sess.Table("pg_indexes").Cols("indexname").Where("tablename = ? ", tableName).Find(&indices); err != nil {
log.Error("Unable to rename %s to %s. Error: %v", tempTableName, tableName, err) log.Error("Unable to rename %s to %s. Error: %v", tempTableName, tableName, err)
@@ -545,6 +569,43 @@ func recreateTable(sess *xorm.Session, bean interface{}) error {
} }
} }
var sequences []string
sess.Engine().SetSchema("")
if err := sess.Table("information_schema.sequences").Cols("sequence_name").Where("sequence_name LIKE 'tmp_recreate__' || ? || '_%' AND sequence_catalog = ?", tableName, setting.Database.Name).Find(&sequences); err != nil {
log.Error("Unable to rename %s to %s. Error: %v", tempTableName, tableName, err)
return err
}
sess.Engine().SetSchema(schema)
for _, sequence := range sequences {
newSequenceName := strings.Replace(sequence, "tmp_recreate__", "", 1)
if _, err := sess.Exec(fmt.Sprintf("ALTER SEQUENCE `%s` RENAME TO `%s`", sequence, newSequenceName)); err != nil {
log.Error("Unable to rename %s sequence to %s. Error: %v", sequence, newSequenceName, err)
return err
}
val, ok := sequenceMap[newSequenceName]
if newSequenceName == tableName+"_id_seq" {
if ok && val.LastValue != 0 {
if _, err := sess.Exec(fmt.Sprintf("SELECT setval('%s', %d, %t)", newSequenceName, val.LastValue, val.IsCalled)); err != nil {
log.Error("Unable to reset %s to %d. Error: %v", newSequenceName, val, err)
return err
}
} else {
// We're going to try to guess this
if _, err := sess.Exec(fmt.Sprintf("SELECT setval('%s', COALESCE((SELECT MAX(id)+1 FROM `%s`), 1), false)", newSequenceName, tableName)); err != nil {
log.Error("Unable to reset %s. Error: %v", newSequenceName, err)
return err
}
}
} else if ok {
if _, err := sess.Exec(fmt.Sprintf("SELECT setval('%s', %d, %t)", newSequenceName, val.LastValue, val.IsCalled)); err != nil {
log.Error("Unable to reset %s to %d. Error: %v", newSequenceName, val, err)
return err
}
}
}
case setting.Database.UseMSSQL: case setting.Database.UseMSSQL:
// MSSQL will drop all the constraints on the old table // MSSQL will drop all the constraints on the old table
if _, err := sess.Exec(fmt.Sprintf("DROP TABLE `%s`", tableName)); err != nil { if _, err := sess.Exec(fmt.Sprintf("DROP TABLE `%s`", tableName)); err != nil {

View File

@@ -1729,6 +1729,10 @@ func DeleteRepository(doer *User, uid, repoID int64) error {
return fmt.Errorf("deleteBeans: %v", err) return fmt.Errorf("deleteBeans: %v", err)
} }
if err := deleteLabelsByRepoID(sess, repoID); err != nil {
return err
}
// Delete Issues and related objects // Delete Issues and related objects
var attachmentPaths []string var attachmentPaths []string
if attachmentPaths, err = deleteIssuesByRepoID(sess, repoID); err != nil { if attachmentPaths, err = deleteIssuesByRepoID(sess, repoID); err != nil {

View File

@@ -727,6 +727,7 @@ var (
"assets", "assets",
"attachments", "attachments",
"avatars", "avatars",
"captcha",
"commits", "commits",
"debug", "debug",
"error", "error",

View File

@@ -13,6 +13,10 @@ import (
// ToUser convert models.User to api.User // ToUser convert models.User to api.User
// signed shall only be set if requester is logged in. authed shall only be set if user is site admin or user himself // signed shall only be set if requester is logged in. authed shall only be set if user is site admin or user himself
func ToUser(user *models.User, signed, authed bool) *api.User { func ToUser(user *models.User, signed, authed bool) *api.User {
if user == nil {
return nil
}
result := &api.User{ result := &api.User{
ID: user.ID, ID: user.ID,
UserName: user.Name, UserName: user.Name,

View File

@@ -30,6 +30,9 @@ var (
// aliasMap provides a map of the alias to its emoji data. // aliasMap provides a map of the alias to its emoji data.
aliasMap map[string]int aliasMap map[string]int
// emptyReplacer is the string replacer for emoji codes.
emptyReplacer *strings.Replacer
// codeReplacer is the string replacer for emoji codes. // codeReplacer is the string replacer for emoji codes.
codeReplacer *strings.Replacer codeReplacer *strings.Replacer
@@ -49,6 +52,7 @@ func loadMap() {
// process emoji codes and aliases // process emoji codes and aliases
codePairs := make([]string, 0) codePairs := make([]string, 0)
emptyPairs := make([]string, 0)
aliasPairs := make([]string, 0) aliasPairs := make([]string, 0)
// sort from largest to small so we match combined emoji first // sort from largest to small so we match combined emoji first
@@ -64,6 +68,7 @@ func loadMap() {
// setup codes // setup codes
codeMap[e.Emoji] = i codeMap[e.Emoji] = i
codePairs = append(codePairs, e.Emoji, ":"+e.Aliases[0]+":") codePairs = append(codePairs, e.Emoji, ":"+e.Aliases[0]+":")
emptyPairs = append(emptyPairs, e.Emoji, e.Emoji)
// setup aliases // setup aliases
for _, a := range e.Aliases { for _, a := range e.Aliases {
@@ -77,6 +82,7 @@ func loadMap() {
} }
// create replacers // create replacers
emptyReplacer = strings.NewReplacer(emptyPairs...)
codeReplacer = strings.NewReplacer(codePairs...) codeReplacer = strings.NewReplacer(codePairs...)
aliasReplacer = strings.NewReplacer(aliasPairs...) aliasReplacer = strings.NewReplacer(aliasPairs...)
}) })
@@ -127,38 +133,53 @@ func ReplaceAliases(s string) string {
return aliasReplacer.Replace(s) return aliasReplacer.Replace(s)
} }
type rememberSecondWriteWriter struct {
pos int
idx int
end int
writecount int
}
func (n *rememberSecondWriteWriter) Write(p []byte) (int, error) {
n.writecount++
if n.writecount == 2 {
n.idx = n.pos
n.end = n.pos + len(p)
}
n.pos += len(p)
return len(p), nil
}
func (n *rememberSecondWriteWriter) WriteString(s string) (int, error) {
n.writecount++
if n.writecount == 2 {
n.idx = n.pos
n.end = n.pos + len(s)
}
n.pos += len(s)
return len(s), nil
}
// FindEmojiSubmatchIndex returns index pair of longest emoji in a string // FindEmojiSubmatchIndex returns index pair of longest emoji in a string
func FindEmojiSubmatchIndex(s string) []int { func FindEmojiSubmatchIndex(s string) []int {
loadMap() loadMap()
found := make(map[int]int) secondWriteWriter := rememberSecondWriteWriter{}
keys := make([]int, 0)
//see if there are any emoji in string before looking for position of specific ones // A faster and clean implementation would copy the trie tree formation in strings.NewReplacer but
//no performance difference when there is a match but 10x faster when there are not // we can be lazy here.
if s == ReplaceCodes(s) { //
// The implementation of strings.Replacer.WriteString is such that the first index of the emoji
// submatch is simply the second thing that is written to WriteString in the writer.
//
// Therefore we can simply take the index of the second write as our first emoji
//
// FIXME: just copy the trie implementation from strings.NewReplacer
_, _ = emptyReplacer.WriteString(&secondWriteWriter, s)
// if we wrote less than twice then we never "replaced"
if secondWriteWriter.writecount < 2 {
return nil return nil
} }
// get index of first emoji occurrence while also checking for longest combination return []int{secondWriteWriter.idx, secondWriteWriter.end}
for j := range GemojiData {
i := strings.Index(s, GemojiData[j].Emoji)
if i != -1 {
if _, ok := found[i]; !ok {
if len(keys) == 0 || i < keys[0] {
found[i] = j
keys = []int{i}
}
if i == 0 {
break
}
}
}
}
if len(keys) > 0 {
index := keys[0]
return []int{index, index + len(GemojiData[found[index]].Emoji)}
}
return nil
} }

View File

@@ -8,6 +8,8 @@ package emoji
import ( import (
"reflect" "reflect"
"testing" "testing"
"github.com/stretchr/testify/assert"
) )
func TestDumpInfo(t *testing.T) { func TestDumpInfo(t *testing.T) {
@@ -65,3 +67,34 @@ func TestReplacers(t *testing.T) {
} }
} }
} }
func TestFindEmojiSubmatchIndex(t *testing.T) {
type testcase struct {
teststring string
expected []int
}
testcases := []testcase{
{
"\U0001f44d",
[]int{0, len("\U0001f44d")},
},
{
"\U0001f44d +1 \U0001f44d \U0001f37a",
[]int{0, 4},
},
{
" \U0001f44d",
[]int{1, 1 + len("\U0001f44d")},
},
{
string([]byte{'\u0001'}) + "\U0001f44d",
[]int{1, 1 + len("\U0001f44d")},
},
}
for _, kase := range testcases {
actual := FindEmojiSubmatchIndex(kase.teststring)
assert.Equal(t, kase.expected, actual)
}
}

View File

@@ -298,19 +298,27 @@ func RenderEmoji(
return ctx.postProcess(rawHTML) return ctx.postProcess(rawHTML)
} }
var tagCleaner = regexp.MustCompile(`<((?:/?\w+/\w+)|(?:/[\w ]+/)|(/?[hH][tT][mM][lL][ />])|(/?[hH][eE][aA][dD][ />]))`)
var nulCleaner = strings.NewReplacer("\000", "")
func (ctx *postProcessCtx) postProcess(rawHTML []byte) ([]byte, error) { func (ctx *postProcessCtx) postProcess(rawHTML []byte) ([]byte, error) {
if ctx.procs == nil { if ctx.procs == nil {
ctx.procs = defaultProcessors ctx.procs = defaultProcessors
} }
// give a generous extra 50 bytes // give a generous extra 50 bytes
res := make([]byte, 0, len(rawHTML)+50) res := bytes.NewBuffer(make([]byte, 0, len(rawHTML)+50))
res = append(res, "<html><body>"...) // prepend "<html><body>"
res = append(res, rawHTML...) _, _ = res.WriteString("<html><body>")
res = append(res, "</body></html>"...)
// Strip out nuls - they're always invalid
_, _ = res.Write(tagCleaner.ReplaceAll([]byte(nulCleaner.Replace(string(rawHTML))), []byte("&lt;$1")))
// close the tags
_, _ = res.WriteString("</body></html>")
// parse the HTML // parse the HTML
nodes, err := html.ParseFragment(bytes.NewReader(res), nil) nodes, err := html.ParseFragment(res, nil)
if err != nil { if err != nil {
return nil, &postProcessError{"invalid HTML", err} return nil, &postProcessError{"invalid HTML", err}
} }
@@ -347,17 +355,17 @@ func (ctx *postProcessCtx) postProcess(rawHTML []byte) ([]byte, error) {
// Create buffer in which the data will be placed again. We know that the // Create buffer in which the data will be placed again. We know that the
// length will be at least that of res; to spare a few alloc+copy, we // length will be at least that of res; to spare a few alloc+copy, we
// reuse res, resetting its length to 0. // reuse res, resetting its length to 0.
buf := bytes.NewBuffer(res[:0]) res.Reset()
// Render everything to buf. // Render everything to buf.
for _, node := range nodes { for _, node := range nodes {
err = html.Render(buf, node) err = html.Render(res, node)
if err != nil { if err != nil {
return nil, &postProcessError{"error rendering processed HTML", err} return nil, &postProcessError{"error rendering processed HTML", err}
} }
} }
// Everything done successfully, return parsed data. // Everything done successfully, return parsed data.
return buf.Bytes(), nil return res.Bytes(), nil
} }
func (ctx *postProcessCtx) visitNode(node *html.Node, visitText bool) { func (ctx *postProcessCtx) visitNode(node *html.Node, visitText bool) {

View File

@@ -10,6 +10,7 @@ import (
"regexp" "regexp"
"strings" "strings"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/markup" "code.gitea.io/gitea/modules/markup"
"code.gitea.io/gitea/modules/markup/common" "code.gitea.io/gitea/modules/markup/common"
"code.gitea.io/gitea/modules/setting" "code.gitea.io/gitea/modules/setting"
@@ -76,6 +77,12 @@ func (g *ASTTransformer) Transform(node *ast.Document, reader text.Reader, pc pa
header.ID = util.BytesToReadOnlyString(id.([]byte)) header.ID = util.BytesToReadOnlyString(id.([]byte))
} }
toc = append(toc, header) toc = append(toc, header)
} else {
for _, attr := range v.Attributes() {
if _, ok := attr.Value.([]byte); !ok {
v.SetAttribute(attr.Name, []byte(fmt.Sprintf("%v", attr.Value)))
}
}
} }
case *ast.Image: case *ast.Image:
// Images need two things: // Images need two things:
@@ -101,11 +108,41 @@ func (g *ASTTransformer) Transform(node *ast.Document, reader text.Reader, pc pa
parent := n.Parent() parent := n.Parent()
// Create a link around image only if parent is not already a link // Create a link around image only if parent is not already a link
if _, ok := parent.(*ast.Link); !ok && parent != nil { if _, ok := parent.(*ast.Link); !ok && parent != nil {
next := n.NextSibling()
// Create a link wrapper
wrap := ast.NewLink() wrap := ast.NewLink()
wrap.Destination = link wrap.Destination = link
wrap.Title = v.Title wrap.Title = v.Title
// Duplicate the current image node
image := ast.NewImage(ast.NewLink())
image.Destination = link
image.Title = v.Title
for _, attr := range v.Attributes() {
image.SetAttribute(attr.Name, attr.Value)
}
for child := v.FirstChild(); child != nil; {
next := child.NextSibling()
image.AppendChild(image, child)
child = next
}
// Append our duplicate image to the wrapper link
wrap.AppendChild(wrap, image)
// Wire in the next sibling
wrap.SetNextSibling(next)
// Replace the current node with the wrapper link
parent.ReplaceChild(parent, n, wrap) parent.ReplaceChild(parent, n, wrap)
wrap.AppendChild(wrap, n)
// But most importantly ensure the next sibling is still on the old image too
v.SetNextSibling(next)
} else {
log.Debug("ast.Image: %s has parent: %v", link, parent)
} }
case *ast.Link: case *ast.Link:
// Links need their href to munged to be a real value // Links need their href to munged to be a real value

View File

@@ -6,7 +6,8 @@
package markdown package markdown
import ( import (
"bytes" "fmt"
"io"
"strings" "strings"
"sync" "sync"
@@ -18,7 +19,7 @@ import (
chromahtml "github.com/alecthomas/chroma/formatters/html" chromahtml "github.com/alecthomas/chroma/formatters/html"
"github.com/yuin/goldmark" "github.com/yuin/goldmark"
"github.com/yuin/goldmark-highlighting" highlighting "github.com/yuin/goldmark-highlighting"
meta "github.com/yuin/goldmark-meta" meta "github.com/yuin/goldmark-meta"
"github.com/yuin/goldmark/extension" "github.com/yuin/goldmark/extension"
"github.com/yuin/goldmark/parser" "github.com/yuin/goldmark/parser"
@@ -34,6 +35,44 @@ var urlPrefixKey = parser.NewContextKey()
var isWikiKey = parser.NewContextKey() var isWikiKey = parser.NewContextKey()
var renderMetasKey = parser.NewContextKey() var renderMetasKey = parser.NewContextKey()
type closesWithError interface {
io.WriteCloser
CloseWithError(err error) error
}
type limitWriter struct {
w closesWithError
sum int64
limit int64
}
// Write implements the standard Write interface:
func (l *limitWriter) Write(data []byte) (int, error) {
leftToWrite := l.limit - l.sum
if leftToWrite < int64(len(data)) {
n, err := l.w.Write(data[:leftToWrite])
l.sum += int64(n)
if err != nil {
return n, err
}
_ = l.w.Close()
return n, fmt.Errorf("Rendered content too large - truncating render")
}
n, err := l.w.Write(data)
l.sum += int64(n)
return n, err
}
// Close closes the writer
func (l *limitWriter) Close() error {
return l.w.Close()
}
// CloseWithError closes the writer
func (l *limitWriter) CloseWithError(err error) error {
return l.w.CloseWithError(err)
}
// NewGiteaParseContext creates a parser.Context with the gitea context set // NewGiteaParseContext creates a parser.Context with the gitea context set
func NewGiteaParseContext(urlPrefix string, metas map[string]string, isWiki bool) parser.Context { func NewGiteaParseContext(urlPrefix string, metas map[string]string, isWiki bool) parser.Context {
pc := parser.NewContext(parser.WithIDs(newPrefixedIDs())) pc := parser.NewContext(parser.WithIDs(newPrefixedIDs()))
@@ -43,8 +82,8 @@ func NewGiteaParseContext(urlPrefix string, metas map[string]string, isWiki bool
return pc return pc
} }
// render renders Markdown to HTML without handling special links. // actualRender renders Markdown to HTML without handling special links.
func render(body []byte, urlPrefix string, metas map[string]string, wikiMarkdown bool) []byte { func actualRender(body []byte, urlPrefix string, metas map[string]string, wikiMarkdown bool) []byte {
once.Do(func() { once.Do(func() {
converter = goldmark.New( converter = goldmark.New(
goldmark.WithExtensions(extension.Table, goldmark.WithExtensions(extension.Table,
@@ -119,12 +158,57 @@ func render(body []byte, urlPrefix string, metas map[string]string, wikiMarkdown
}) })
pc := NewGiteaParseContext(urlPrefix, metas, wikiMarkdown) rd, wr := io.Pipe()
var buf bytes.Buffer defer func() {
if err := converter.Convert(giteautil.NormalizeEOL(body), &buf, parser.WithContext(pc)); err != nil { _ = rd.Close()
log.Error("Unable to render: %v", err) _ = wr.Close()
}()
lw := &limitWriter{
w: wr,
limit: setting.UI.MaxDisplayFileSize * 3,
} }
return markup.SanitizeReader(&buf).Bytes()
// FIXME: should we include a timeout that closes the pipe to abort the parser and sanitizer if it takes too long?
go func() {
defer func() {
err := recover()
if err == nil {
return
}
log.Warn("Unable to render markdown due to panic in goldmark: %v", err)
if log.IsDebug() {
log.Debug("Panic in markdown: %v\n%s", err, string(log.Stack(2)))
}
_ = lw.CloseWithError(fmt.Errorf("%v", err))
}()
pc := NewGiteaParseContext(urlPrefix, metas, wikiMarkdown)
if err := converter.Convert(giteautil.NormalizeEOL(body), lw, parser.WithContext(pc)); err != nil {
log.Error("Unable to render: %v", err)
_ = lw.CloseWithError(err)
return
}
_ = lw.Close()
}()
return markup.SanitizeReader(rd).Bytes()
}
func render(body []byte, urlPrefix string, metas map[string]string, wikiMarkdown bool) (ret []byte) {
defer func() {
err := recover()
if err == nil {
return
}
log.Warn("Unable to render markdown due to panic in goldmark - will return sanitized raw bytes")
if log.IsDebug() {
log.Debug("Panic in markdown: %v\n%s", err, string(log.Stack(2)))
}
ret = markup.SanitizeBytes(body)
}()
return actualRender(body, urlPrefix, metas, wikiMarkdown)
} }
var ( var (

View File

@@ -308,3 +308,34 @@ func TestRender_RenderParagraphs(t *testing.T) {
test(t, "A\n\nB\nC\n", 2) test(t, "A\n\nB\nC\n", 2)
test(t, "A\n\n\nB\nC\n", 2) test(t, "A\n\n\nB\nC\n", 2)
} }
func TestMarkdownRenderRaw(t *testing.T) {
testcases := [][]byte{
{ // clusterfuzz_testcase_minimized_fuzz_markdown_render_raw_6267570554535936
0x2a, 0x20, 0x2d, 0x0a, 0x09, 0x20, 0x60, 0x5b, 0x0a, 0x09, 0x20, 0x60,
0x5b,
},
{ // clusterfuzz_testcase_minimized_fuzz_markdown_render_raw_6278827345051648
0x2d, 0x20, 0x2d, 0x0d, 0x09, 0x60, 0x0d, 0x09, 0x60,
},
{ // clusterfuzz_testcase_minimized_fuzz_markdown_render_raw_6016973788020736[] = {
0x7b, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x3d, 0x35, 0x7d, 0x0a, 0x3d,
},
}
for _, testcase := range testcases {
_ = RenderRaw(testcase, "", false)
}
}
func TestRenderSiblingImages_Issue12925(t *testing.T) {
testcase := `![image1](/image1)
![image2](/image2)
`
expected := `<p><a href="/image1" rel="nofollow"><img src="/image1" alt="image1"></a><br>
<a href="/image2" rel="nofollow"><img src="/image2" alt="image2"></a></p>
`
res := string(RenderRaw([]byte(testcase), "", false))
assert.Equal(t, expected, res)
}

View File

@@ -52,6 +52,13 @@ func isMigrateURLAllowed(remoteURL string) error {
} }
} }
if u.Host == "" {
if !setting.ImportLocalPaths {
return &models.ErrMigrationNotAllowed{Host: "<LOCAL_FILESYSTEM>"}
}
return nil
}
if !setting.Migrations.AllowLocalNetworks { if !setting.Migrations.AllowLocalNetworks {
addrList, err := net.LookupIP(strings.Split(u.Host, ":")[0]) addrList, err := net.LookupIP(strings.Split(u.Host, ":")[0])
if err != nil { if err != nil {

View File

@@ -31,4 +31,16 @@ func TestMigrateWhiteBlocklist(t *testing.T) {
err = isMigrateURLAllowed("https://github.com/go-gitea/gitea.git") err = isMigrateURLAllowed("https://github.com/go-gitea/gitea.git")
assert.Error(t, err) assert.Error(t, err)
old := setting.ImportLocalPaths
setting.ImportLocalPaths = false
err = isMigrateURLAllowed("/home/foo/bar/goo")
assert.Error(t, err)
setting.ImportLocalPaths = true
err = isMigrateURLAllowed("/home/foo/bar/goo")
assert.NoError(t, err)
setting.ImportLocalPaths = old
} }

View File

@@ -120,7 +120,6 @@ func UploadRepoFiles(repo *models.Repository, doer *models.User, opts *UploadRep
return err return err
} }
infos[i] = uploadInfo infos[i] = uploadInfo
} else if objectHash, err = t.HashObject(file); err != nil { } else if objectHash, err = t.HashObject(file); err != nil {
return err return err
} }
@@ -128,7 +127,6 @@ func UploadRepoFiles(repo *models.Repository, doer *models.User, opts *UploadRep
// Add the object to the index // Add the object to the index
if err := t.AddObjectToIndex("100644", objectHash, path.Join(opts.TreePath, uploadInfo.upload.Name)); err != nil { if err := t.AddObjectToIndex("100644", objectHash, path.Join(opts.TreePath, uploadInfo.upload.Name)); err != nil {
return err return err
} }
} }
@@ -165,28 +163,10 @@ func UploadRepoFiles(repo *models.Repository, doer *models.User, opts *UploadRep
// OK now we can insert the data into the store - there's no way to clean up the store // OK now we can insert the data into the store - there's no way to clean up the store
// once it's in there, it's in there. // once it's in there, it's in there.
contentStore := &lfs.ContentStore{ObjectStorage: storage.LFS} contentStore := &lfs.ContentStore{ObjectStorage: storage.LFS}
for _, uploadInfo := range infos { for _, info := range infos {
if uploadInfo.lfsMetaObject == nil { if err := uploadToLFSContentStore(info, contentStore); err != nil {
continue
}
exist, err := contentStore.Exists(uploadInfo.lfsMetaObject)
if err != nil {
return cleanUpAfterFailure(&infos, t, err) return cleanUpAfterFailure(&infos, t, err)
} }
if !exist {
file, err := os.Open(uploadInfo.upload.LocalPath())
if err != nil {
return cleanUpAfterFailure(&infos, t, err)
}
defer file.Close()
// FIXME: Put regenerates the hash and copies the file over.
// I guess this strictly ensures the soundness of the store but this is inefficient.
if err := contentStore.Put(uploadInfo.lfsMetaObject, file); err != nil {
// OK Now we need to cleanup
// Can't clean up the store, once uploaded there they're there.
return cleanUpAfterFailure(&infos, t, err)
}
}
} }
// Then push this tree to NewBranch // Then push this tree to NewBranch
@@ -196,3 +176,29 @@ func UploadRepoFiles(repo *models.Repository, doer *models.User, opts *UploadRep
return models.DeleteUploads(uploads...) return models.DeleteUploads(uploads...)
} }
func uploadToLFSContentStore(info uploadInfo, contentStore *lfs.ContentStore) error {
if info.lfsMetaObject == nil {
return nil
}
exist, err := contentStore.Exists(info.lfsMetaObject)
if err != nil {
return err
}
if !exist {
file, err := os.Open(info.upload.LocalPath())
if err != nil {
return err
}
defer file.Close()
// FIXME: Put regenerates the hash and copies the file over.
// I guess this strictly ensures the soundness of the store but this is inefficient.
if err := contentStore.Put(info.lfsMetaObject, file); err != nil {
// OK Now we need to cleanup
// Can't clean up the store, once uploaded there they're there.
return err
}
}
return nil
}

View File

@@ -293,7 +293,6 @@ func CreatePullRequest(ctx *context.APIContext, form api.CreatePullRequestOption
var ( var (
repo = ctx.Repo.Repository repo = ctx.Repo.Repository
labelIDs []int64 labelIDs []int64
assigneeID int64
milestoneID int64 milestoneID int64
) )
@@ -354,7 +353,7 @@ func CreatePullRequest(ctx *context.APIContext, form api.CreatePullRequestOption
} }
if form.Milestone > 0 { if form.Milestone > 0 {
milestone, err := models.GetMilestoneByRepoID(ctx.Repo.Repository.ID, milestoneID) milestone, err := models.GetMilestoneByRepoID(ctx.Repo.Repository.ID, form.Milestone)
if err != nil { if err != nil {
if models.IsErrMilestoneNotExist(err) { if models.IsErrMilestoneNotExist(err) {
ctx.NotFound() ctx.NotFound()
@@ -378,7 +377,6 @@ func CreatePullRequest(ctx *context.APIContext, form api.CreatePullRequestOption
PosterID: ctx.User.ID, PosterID: ctx.User.ID,
Poster: ctx.User, Poster: ctx.User,
MilestoneID: milestoneID, MilestoneID: milestoneID,
AssigneeID: assigneeID,
IsPull: true, IsPull: true,
Content: form.Body, Content: form.Body,
DeadlineUnix: deadlineUnix, DeadlineUnix: deadlineUnix,

View File

@@ -539,6 +539,10 @@ func updateBasicProperties(ctx *context.APIContext, opts api.EditRepoOption) err
if opts.Private != nil { if opts.Private != nil {
// Visibility of forked repository is forced sync with base repository. // Visibility of forked repository is forced sync with base repository.
if repo.IsFork { if repo.IsFork {
if err := repo.GetBaseRepo(); err != nil {
ctx.Error(http.StatusInternalServerError, "Unable to load base repository", err)
return err
}
*opts.Private = repo.BaseRepo.IsPrivate *opts.Private = repo.BaseRepo.IsPrivate
} }

View File

@@ -13,6 +13,7 @@ import (
"net/http" "net/http"
"os" "os"
"path" "path"
"path/filepath"
"strings" "strings"
"text/template" "text/template"
"time" "time"
@@ -152,12 +153,21 @@ func storageHandler(storageSetting setting.Storage, prefix string, objStore stor
return return
} }
if !strings.HasPrefix(req.URL.RequestURI(), "/"+prefix) { prefix := strings.Trim(prefix, "/")
if !strings.HasPrefix(req.URL.EscapedPath(), "/"+prefix+"/") {
return return
} }
rPath := strings.TrimPrefix(req.URL.EscapedPath(), "/"+prefix+"/")
rPath := strings.TrimPrefix(req.URL.RequestURI(), "/"+prefix)
rPath = strings.TrimPrefix(rPath, "/") rPath = strings.TrimPrefix(rPath, "/")
if rPath == "" {
ctx.Error(404, "file not found")
return
}
rPath = path.Clean("/" + filepath.ToSlash(rPath))
rPath = rPath[1:]
//If we have matched and access to release or issue //If we have matched and access to release or issue
fr, err := objStore.Open(rPath) fr, err := objStore.Open(rPath)
if err != nil { if err != nil {

View File

@@ -1282,6 +1282,16 @@ func CommentAsDiff(c *models.Comment) (*Diff, error) {
// CommentMustAsDiff executes AsDiff and logs the error instead of returning // CommentMustAsDiff executes AsDiff and logs the error instead of returning
func CommentMustAsDiff(c *models.Comment) *Diff { func CommentMustAsDiff(c *models.Comment) *Diff {
if c == nil {
return nil
}
defer func() {
if err := recover(); err != nil {
stack := log.Stack(2)
log.Error("PANIC whilst retrieving diff for comment[%d] Error: %v\nStack: %s", c.ID, err, stack)
panic(fmt.Errorf("PANIC whilst retrieving diff for comment[%d] Error: %v\nStack: %s", c.ID, err, stack))
}
}()
diff, err := CommentAsDiff(c) diff, err := CommentAsDiff(c)
if err != nil { if err != nil {
log.Warn("CommentMustAsDiff: %v", err) log.Warn("CommentMustAsDiff: %v", err)

View File

@@ -12,8 +12,10 @@
</head> </head>
<body> <body>
<p><b>@{{.Release.Publisher.Name}}</b> released <a href="{{.Release.HTMLURL}}">{{.Release.TagName}}</a> <p>
in <a href="{{AppUrl}}{{.Release.Repo.OwnerName}}/{{.Release.Repo.Name}}">{{.Release.Repo.FullName}} </p> <b>@{{.Release.Publisher.Name}}</b> released <a href="{{.Release.HTMLURL}}">{{.Release.TagName}}</a>
in <a href="{{AppUrl}}{{.Release.Repo.OwnerName}}/{{.Release.Repo.Name}}">{{.Release.Repo.FullName}}</a>
</p>
<h4>Title: {{.Release.Title}}</h4> <h4>Title: {{.Release.Title}}</h4>
<p> <p>
Note: <br> Note: <br>

View File

@@ -4,8 +4,8 @@
<div class="ui container"> <div class="ui container">
<div class="ui three column stackable grid"> <div class="ui three column stackable grid">
<div class="column"> <div class="column">
<h3>{{.Milestone.Name}}</h3> <h1>{{.Milestone.Name}}</h1>
<div class="content"> <div class="markdown content">
{{.Milestone.RenderedContent|Str2html}} {{.Milestone.RenderedContent|Str2html}}
</div> </div>
</div> </div>
@@ -63,7 +63,7 @@
<span class="info">{{.i18n.Tr "repo.issues.filter_label_exclude" | Safe}}</span> <span class="info">{{.i18n.Tr "repo.issues.filter_label_exclude" | Safe}}</span>
<a class="item" href="{{$.Link}}?q={{$.Keyword}}&type={{$.ViewType}}&sort={{$.SortType}}&state={{$.State}}&assignee={{$.AssigneeID}}">{{.i18n.Tr "repo.issues.filter_label_no_select"}}</a> <a class="item" href="{{$.Link}}?q={{$.Keyword}}&type={{$.ViewType}}&sort={{$.SortType}}&state={{$.State}}&assignee={{$.AssigneeID}}">{{.i18n.Tr "repo.issues.filter_label_no_select"}}</a>
{{range .Labels}} {{range .Labels}}
<a class="item label-filter-item" href="{{$.Link}}?q={{$.Keyword}}&type={{$.ViewType}}&sort={{$.SortType}}&state={{$.State}}&labels={{.ID}}&assignee={{$.AssigneeID}}" data-label-id="{{.ID}}">{{if .IsExcluded}}{{svg "octicon-circle-slash"}}{{else if contain $.SelLabelIDs .ID}}{{svg "octicon-check"}}{{end}}<span class="label color" style="background-color: {{.Color}}"></span> {{.Name | RenderEmoji}}</a> <a class="item label-filter-item" href="{{$.Link}}?q={{$.Keyword}}&type={{$.ViewType}}&sort={{$.SortType}}&state={{$.State}}&labels={{.QueryString}}&assignee={{$.AssigneeID}}" data-label-id="{{.ID}}">{{if .IsExcluded}}{{svg "octicon-circle-slash"}}{{else if contain $.SelLabelIDs .ID}}{{svg "octicon-check"}}{{end}}<span class="label color" style="background-color: {{.Color}}"></span> {{.Name | RenderEmoji}}</a>
{{end}} {{end}}
</div> </div>
</div> </div>

View File

@@ -43,7 +43,7 @@
<div class="milestone list"> <div class="milestone list">
{{range .Milestones}} {{range .Milestones}}
<li class="item"> <li class="item">
{{svg "octicon-milestone"}} <a href="{{$.RepoLink}}/milestone/{{.ID}}">{{.Name}}</a> {{svg "octicon-milestone" 16 "mr-2"}} <a href="{{$.RepoLink}}/milestone/{{.ID}}">{{.Name}}</a>
<div class="ui right green progress" data-percent="{{.Completeness}}"> <div class="ui right green progress" data-percent="{{.Completeness}}">
<div class="bar" {{if not .Completeness}}style="background-color: transparent"{{end}}> <div class="bar" {{if not .Completeness}}style="background-color: transparent"{{end}}>
<div class="progress"></div> <div class="progress"></div>
@@ -80,7 +80,7 @@
</div> </div>
{{end}} {{end}}
{{if .Content}} {{if .Content}}
<div class="content"> <div class="markdown content">
{{.RenderedContent|Str2html}} {{.RenderedContent|Str2html}}
</div> </div>
{{end}} {{end}}

View File

@@ -29,7 +29,7 @@
</div> </div>
<div class="header-right actions df ac"> <div class="header-right actions df ac">
{{if not $.Repository.IsArchived}} {{if not $.Repository.IsArchived}}
{{if or (and (eq .PosterID .Issue.PosterID) (eq .Issue.OriginalAuthorID 0)) (eq .Issue.OriginalAuthorID .OriginalAuthorID) }} {{if or (and (eq .PosterID .Issue.PosterID) (eq .Issue.OriginalAuthorID 0)) (and (eq .Issue.OriginalAuthorID .OriginalAuthorID) (not (eq .OriginalAuthorID 0))) }}
<div class="item tag"> <div class="item tag">
{{$.i18n.Tr "repo.issues.poster"}} {{$.i18n.Tr "repo.issues.poster"}}
</div> </div>

View File

@@ -1,7 +1,7 @@
goldmark goldmark
========================================== ==========================================
[![http://godoc.org/github.com/yuin/goldmark](https://godoc.org/github.com/yuin/goldmark?status.svg)](http://godoc.org/github.com/yuin/goldmark) [![https://pkg.go.dev/github.com/yuin/goldmark](https://pkg.go.dev/badge/github.com/yuin/goldmark.svg)](https://pkg.go.dev/github.com/yuin/goldmark)
[![https://github.com/yuin/goldmark/actions?query=workflow:test](https://github.com/yuin/goldmark/workflows/test/badge.svg?branch=master&event=push)](https://github.com/yuin/goldmark/actions?query=workflow:test) [![https://github.com/yuin/goldmark/actions?query=workflow:test](https://github.com/yuin/goldmark/workflows/test/badge.svg?branch=master&event=push)](https://github.com/yuin/goldmark/actions?query=workflow:test)
[![https://coveralls.io/github/yuin/goldmark](https://coveralls.io/repos/github/yuin/goldmark/badge.svg?branch=master)](https://coveralls.io/github/yuin/goldmark) [![https://coveralls.io/github/yuin/goldmark](https://coveralls.io/repos/github/yuin/goldmark/badge.svg?branch=master)](https://coveralls.io/github/yuin/goldmark)
[![https://goreportcard.com/report/github.com/yuin/goldmark](https://goreportcard.com/badge/github.com/yuin/goldmark)](https://goreportcard.com/report/github.com/yuin/goldmark) [![https://goreportcard.com/report/github.com/yuin/goldmark](https://goreportcard.com/badge/github.com/yuin/goldmark)](https://goreportcard.com/report/github.com/yuin/goldmark)
@@ -173,6 +173,7 @@ Parser and Renderer options
- This extension enables Table, Strikethrough, Linkify and TaskList. - This extension enables Table, Strikethrough, Linkify and TaskList.
- This extension does not filter tags defined in [6.11: Disallowed Raw HTML (extension)](https://github.github.com/gfm/#disallowed-raw-html-extension-). - This extension does not filter tags defined in [6.11: Disallowed Raw HTML (extension)](https://github.github.com/gfm/#disallowed-raw-html-extension-).
If you need to filter HTML tags, see [Security](#security). If you need to filter HTML tags, see [Security](#security).
- If you need to parse github emojis, you can use [goldmark-emoji](https://github.com/yuin/goldmark-emoji) extension.
- `extension.DefinitionList` - `extension.DefinitionList`
- [PHP Markdown Extra: Definition lists](https://michelf.ca/projects/php-markdown/extra/#def-list) - [PHP Markdown Extra: Definition lists](https://michelf.ca/projects/php-markdown/extra/#def-list)
- `extension.Footnote` - `extension.Footnote`
@@ -279,13 +280,96 @@ markdown := goldmark.New(
[]byte("https:"), []byte("https:"),
}), }),
extension.WithLinkifyURLRegexp( extension.WithLinkifyURLRegexp(
xurls.Strict(), xurls.Strict,
), ),
), ),
), ),
) )
``` ```
### Footnotes extension
The Footnote extension implements [PHP Markdown Extra: Footnotes](https://michelf.ca/projects/php-markdown/extra/#footnotes).
This extension has some options:
| Functional option | Type | Description |
| ----------------- | ---- | ----------- |
| `extension.WithFootnoteIDPrefix` | `[]byte` | a prefix for the id attributes.|
| `extension.WithFootnoteIDPrefixFunction` | `func(gast.Node) []byte` | a function that determines the id attribute for given Node.|
| `extension.WithFootnoteLinkTitle` | `[]byte` | an optional title attribute for footnote links.|
| `extension.WithFootnoteBacklinkTitle` | `[]byte` | an optional title attribute for footnote backlinks. |
| `extension.WithFootnoteLinkClass` | `[]byte` | a class for footnote links. This defaults to `footnote-ref`. |
| `extension.WithFootnoteBacklinkClass` | `[]byte` | a class for footnote backlinks. This defaults to `footnote-backref`. |
| `extension.WithFootnoteBacklinkHTML` | `[]byte` | a class for footnote backlinks. This defaults to `&#x21a9;&#xfe0e;`. |
Some options can have special substitutions. Occurances of “^^” in the string will be replaced by the corresponding footnote number in the HTML output. Occurances of “%%” will be replaced by a number for the reference (footnotes can have multiple references).
`extension.WithFootnoteIDPrefix` and `extension.WithFootnoteIDPrefixFunction` are useful if you have multiple Markdown documents displayed inside one HTML document to avoid footnote ids to clash each other.
`extension.WithFootnoteIDPrefix` sets fixed id prefix, so you may write codes like the following:
```go
for _, path := range files {
source := readAll(path)
prefix := getPrefix(path)
markdown := goldmark.New(
goldmark.WithExtensions(
NewFootnote(
WithFootnoteIDPrefix([]byte(path)),
),
),
)
var b bytes.Buffer
err := markdown.Convert(source, &b)
if err != nil {
t.Error(err.Error())
}
}
```
`extension.WithFootnoteIDPrefixFunction` determines an id prefix by calling given function, so you may write codes like the following:
```go
markdown := goldmark.New(
goldmark.WithExtensions(
NewFootnote(
WithFootnoteIDPrefixFunction(func(n gast.Node) []byte {
v, ok := n.OwnerDocument().Meta()["footnote-prefix"]
if ok {
return util.StringToReadOnlyBytes(v.(string))
}
return nil
}),
),
),
)
for _, path := range files {
source := readAll(path)
var b bytes.Buffer
doc := markdown.Parser().Parse(text.NewReader(source))
doc.Meta()["footnote-prefix"] = getPrefix(path)
err := markdown.Renderer().Render(&b, source, doc)
}
```
You can use [goldmark-meta](https://github.com/yuin/goldmark-meta) to define a id prefix in the markdown document:
```markdown
---
title: document title
slug: article1
footnote-prefix: article1
---
# My article
```
Security Security
-------------------- --------------------
By default, goldmark does not render raw HTML or potentially-dangerous URLs. By default, goldmark does not render raw HTML or potentially-dangerous URLs.
@@ -336,6 +420,8 @@ Extensions
extension for the goldmark Markdown parser. extension for the goldmark Markdown parser.
- [goldmark-highlighting](https://github.com/yuin/goldmark-highlighting): A syntax-highlighting extension - [goldmark-highlighting](https://github.com/yuin/goldmark-highlighting): A syntax-highlighting extension
for the goldmark markdown parser. for the goldmark markdown parser.
- [goldmark-emoji](https://github.com/yuin/goldmark-emoji): An emoji
extension for the goldmark Markdown parser.
- [goldmark-mathjax](https://github.com/litao91/goldmark-mathjax): Mathjax support for the goldmark markdown parser - [goldmark-mathjax](https://github.com/litao91/goldmark-mathjax): Mathjax support for the goldmark markdown parser
goldmark internal(for extension developers) goldmark internal(for extension developers)

View File

@@ -45,11 +45,6 @@ type Attribute struct {
Value interface{} Value interface{}
} }
var attrNameIDS = []byte("#")
var attrNameID = []byte("id")
var attrNameClassS = []byte(".")
var attrNameClass = []byte("class")
// A Node interface defines basic AST node functionalities. // A Node interface defines basic AST node functionalities.
type Node interface { type Node interface {
// Type returns a type of this node. // Type returns a type of this node.
@@ -116,6 +111,11 @@ type Node interface {
// tail of the children. // tail of the children.
InsertAfter(self, v1, insertee Node) InsertAfter(self, v1, insertee Node)
// OwnerDocument returns this node's owner document.
// If this node is not a child of the Document node, OwnerDocument
// returns nil.
OwnerDocument() *Document
// Dump dumps an AST tree structure to stdout. // Dump dumps an AST tree structure to stdout.
// This function completely aimed for debugging. // This function completely aimed for debugging.
// level is a indent level. Implementer should indent informations with // level is a indent level. Implementer should indent informations with
@@ -169,7 +169,7 @@ type Node interface {
RemoveAttributes() RemoveAttributes()
} }
// A BaseNode struct implements the Node interface. // A BaseNode struct implements the Node interface partialliy.
type BaseNode struct { type BaseNode struct {
firstChild Node firstChild Node
lastChild Node lastChild Node
@@ -358,6 +358,22 @@ func (n *BaseNode) InsertBefore(self, v1, insertee Node) {
} }
} }
// OwnerDocument implements Node.OwnerDocument
func (n *BaseNode) OwnerDocument() *Document {
d := n.Parent()
for {
p := d.Parent()
if p == nil {
if v, ok := d.(*Document); ok {
return v
}
break
}
d = p
}
return nil
}
// Text implements Node.Text . // Text implements Node.Text .
func (n *BaseNode) Text(source []byte) []byte { func (n *BaseNode) Text(source []byte) []byte {
var buf bytes.Buffer var buf bytes.Buffer

View File

@@ -7,7 +7,7 @@ import (
textm "github.com/yuin/goldmark/text" textm "github.com/yuin/goldmark/text"
) )
// A BaseBlock struct implements the Node interface. // A BaseBlock struct implements the Node interface partialliy.
type BaseBlock struct { type BaseBlock struct {
BaseNode BaseNode
blankPreviousLines bool blankPreviousLines bool
@@ -50,6 +50,8 @@ func (b *BaseBlock) SetLines(v *textm.Segments) {
// A Document struct is a root node of Markdown text. // A Document struct is a root node of Markdown text.
type Document struct { type Document struct {
BaseBlock BaseBlock
meta map[string]interface{}
} }
// KindDocument is a NodeKind of the Document node. // KindDocument is a NodeKind of the Document node.
@@ -70,10 +72,29 @@ func (n *Document) Kind() NodeKind {
return KindDocument return KindDocument
} }
// OwnerDocument implements Node.OwnerDocument
func (n *Document) OwnerDocument() *Document {
return n
}
// Meta returns metadata of this document.
func (n *Document) Meta() map[string]interface{} {
if n.meta == nil {
n.meta = map[string]interface{}{}
}
return n.meta
}
// SetMeta sets given metadata to this document.
func (n *Document) SetMeta(meta map[string]interface{}) {
n.meta = meta
}
// NewDocument returns a new Document node. // NewDocument returns a new Document node.
func NewDocument() *Document { func NewDocument() *Document {
return &Document{ return &Document{
BaseBlock: BaseBlock{}, BaseBlock: BaseBlock{},
meta: nil,
} }
} }

View File

@@ -8,7 +8,7 @@ import (
"github.com/yuin/goldmark/util" "github.com/yuin/goldmark/util"
) )
// A BaseInline struct implements the Node interface. // A BaseInline struct implements the Node interface partialliy.
type BaseInline struct { type BaseInline struct {
BaseNode BaseNode
} }

View File

@@ -2,6 +2,7 @@ package ast
import ( import (
"fmt" "fmt"
gast "github.com/yuin/goldmark/ast" gast "github.com/yuin/goldmark/ast"
) )
@@ -9,13 +10,15 @@ import (
// (PHP Markdown Extra) text. // (PHP Markdown Extra) text.
type FootnoteLink struct { type FootnoteLink struct {
gast.BaseInline gast.BaseInline
Index int Index int
RefCount int
} }
// Dump implements Node.Dump. // Dump implements Node.Dump.
func (n *FootnoteLink) Dump(source []byte, level int) { func (n *FootnoteLink) Dump(source []byte, level int) {
m := map[string]string{} m := map[string]string{}
m["Index"] = fmt.Sprintf("%v", n.Index) m["Index"] = fmt.Sprintf("%v", n.Index)
m["RefCount"] = fmt.Sprintf("%v", n.RefCount)
gast.DumpHelper(n, source, level, m, nil) gast.DumpHelper(n, source, level, m, nil)
} }
@@ -30,36 +33,40 @@ func (n *FootnoteLink) Kind() gast.NodeKind {
// NewFootnoteLink returns a new FootnoteLink node. // NewFootnoteLink returns a new FootnoteLink node.
func NewFootnoteLink(index int) *FootnoteLink { func NewFootnoteLink(index int) *FootnoteLink {
return &FootnoteLink{ return &FootnoteLink{
Index: index, Index: index,
RefCount: 0,
} }
} }
// A FootnoteBackLink struct represents a link to a footnote of Markdown // A FootnoteBacklink struct represents a link to a footnote of Markdown
// (PHP Markdown Extra) text. // (PHP Markdown Extra) text.
type FootnoteBackLink struct { type FootnoteBacklink struct {
gast.BaseInline gast.BaseInline
Index int Index int
RefCount int
} }
// Dump implements Node.Dump. // Dump implements Node.Dump.
func (n *FootnoteBackLink) Dump(source []byte, level int) { func (n *FootnoteBacklink) Dump(source []byte, level int) {
m := map[string]string{} m := map[string]string{}
m["Index"] = fmt.Sprintf("%v", n.Index) m["Index"] = fmt.Sprintf("%v", n.Index)
m["RefCount"] = fmt.Sprintf("%v", n.RefCount)
gast.DumpHelper(n, source, level, m, nil) gast.DumpHelper(n, source, level, m, nil)
} }
// KindFootnoteBackLink is a NodeKind of the FootnoteBackLink node. // KindFootnoteBacklink is a NodeKind of the FootnoteBacklink node.
var KindFootnoteBackLink = gast.NewNodeKind("FootnoteBackLink") var KindFootnoteBacklink = gast.NewNodeKind("FootnoteBacklink")
// Kind implements Node.Kind. // Kind implements Node.Kind.
func (n *FootnoteBackLink) Kind() gast.NodeKind { func (n *FootnoteBacklink) Kind() gast.NodeKind {
return KindFootnoteBackLink return KindFootnoteBacklink
} }
// NewFootnoteBackLink returns a new FootnoteBackLink node. // NewFootnoteBacklink returns a new FootnoteBacklink node.
func NewFootnoteBackLink(index int) *FootnoteBackLink { func NewFootnoteBacklink(index int) *FootnoteBacklink {
return &FootnoteBackLink{ return &FootnoteBacklink{
Index: index, Index: index,
RefCount: 0,
} }
} }

View File

@@ -2,6 +2,8 @@ package extension
import ( import (
"bytes" "bytes"
"strconv"
"github.com/yuin/goldmark" "github.com/yuin/goldmark"
gast "github.com/yuin/goldmark/ast" gast "github.com/yuin/goldmark/ast"
"github.com/yuin/goldmark/extension/ast" "github.com/yuin/goldmark/extension/ast"
@@ -10,10 +12,10 @@ import (
"github.com/yuin/goldmark/renderer/html" "github.com/yuin/goldmark/renderer/html"
"github.com/yuin/goldmark/text" "github.com/yuin/goldmark/text"
"github.com/yuin/goldmark/util" "github.com/yuin/goldmark/util"
"strconv"
) )
var footnoteListKey = parser.NewContextKey() var footnoteListKey = parser.NewContextKey()
var footnoteLinkListKey = parser.NewContextKey()
type footnoteBlockParser struct { type footnoteBlockParser struct {
} }
@@ -164,7 +166,20 @@ func (s *footnoteParser) Parse(parent gast.Node, block text.Reader, pc parser.Co
return nil return nil
} }
return ast.NewFootnoteLink(index) fnlink := ast.NewFootnoteLink(index)
var fnlist []*ast.FootnoteLink
if tmp := pc.Get(footnoteLinkListKey); tmp != nil {
fnlist = tmp.([]*ast.FootnoteLink)
} else {
fnlist = []*ast.FootnoteLink{}
pc.Set(footnoteLinkListKey, fnlist)
}
pc.Set(footnoteLinkListKey, append(fnlist, fnlink))
if line[0] == '!' {
parent.AppendChild(parent, gast.NewTextSegment(text.NewSegment(segment.Start, segment.Start+1)))
}
return fnlink
} }
type footnoteASTTransformer struct { type footnoteASTTransformer struct {
@@ -180,23 +195,46 @@ func NewFootnoteASTTransformer() parser.ASTTransformer {
func (a *footnoteASTTransformer) Transform(node *gast.Document, reader text.Reader, pc parser.Context) { func (a *footnoteASTTransformer) Transform(node *gast.Document, reader text.Reader, pc parser.Context) {
var list *ast.FootnoteList var list *ast.FootnoteList
if tlist := pc.Get(footnoteListKey); tlist != nil { var fnlist []*ast.FootnoteLink
list = tlist.(*ast.FootnoteList) if tmp := pc.Get(footnoteListKey); tmp != nil {
} else { list = tmp.(*ast.FootnoteList)
}
if tmp := pc.Get(footnoteLinkListKey); tmp != nil {
fnlist = tmp.([]*ast.FootnoteLink)
}
pc.Set(footnoteListKey, nil)
pc.Set(footnoteLinkListKey, nil)
if list == nil {
return return
} }
pc.Set(footnoteListKey, nil)
counter := map[int]int{}
if fnlist != nil {
for _, fnlink := range fnlist {
if fnlink.Index >= 0 {
counter[fnlink.Index]++
}
}
for _, fnlink := range fnlist {
fnlink.RefCount = counter[fnlink.Index]
}
}
for footnote := list.FirstChild(); footnote != nil; { for footnote := list.FirstChild(); footnote != nil; {
var container gast.Node = footnote var container gast.Node = footnote
next := footnote.NextSibling() next := footnote.NextSibling()
if fc := container.LastChild(); fc != nil && gast.IsParagraph(fc) { if fc := container.LastChild(); fc != nil && gast.IsParagraph(fc) {
container = fc container = fc
} }
index := footnote.(*ast.Footnote).Index fn := footnote.(*ast.Footnote)
index := fn.Index
if index < 0 { if index < 0 {
list.RemoveChild(list, footnote) list.RemoveChild(list, footnote)
} else { } else {
container.AppendChild(container, ast.NewFootnoteBackLink(index)) backLink := ast.NewFootnoteBacklink(index)
backLink.RefCount = counter[index]
container.AppendChild(container, backLink)
} }
footnote = next footnote = next
} }
@@ -214,19 +252,250 @@ func (a *footnoteASTTransformer) Transform(node *gast.Document, reader text.Read
node.AppendChild(node, list) node.AppendChild(node, list)
} }
// FootnoteConfig holds configuration values for the footnote extension.
//
// Link* and Backlink* configurations have some variables:
// Occurrances of “^^” in the string will be replaced by the
// corresponding footnote number in the HTML output.
// Occurrances of “%%” will be replaced by a number for the
// reference (footnotes can have multiple references).
type FootnoteConfig struct {
html.Config
// IDPrefix is a prefix for the id attributes generated by footnotes.
IDPrefix []byte
// IDPrefix is a function that determines the id attribute for given Node.
IDPrefixFunction func(gast.Node) []byte
// LinkTitle is an optional title attribute for footnote links.
LinkTitle []byte
// BacklinkTitle is an optional title attribute for footnote backlinks.
BacklinkTitle []byte
// LinkClass is a class for footnote links.
LinkClass []byte
// BacklinkClass is a class for footnote backlinks.
BacklinkClass []byte
// BacklinkHTML is an HTML content for footnote backlinks.
BacklinkHTML []byte
}
// FootnoteOption interface is a functional option interface for the extension.
type FootnoteOption interface {
renderer.Option
// SetFootnoteOption sets given option to the extension.
SetFootnoteOption(*FootnoteConfig)
}
// NewFootnoteConfig returns a new Config with defaults.
func NewFootnoteConfig() FootnoteConfig {
return FootnoteConfig{
Config: html.NewConfig(),
LinkTitle: []byte(""),
BacklinkTitle: []byte(""),
LinkClass: []byte("footnote-ref"),
BacklinkClass: []byte("footnote-backref"),
BacklinkHTML: []byte("&#x21a9;&#xfe0e;"),
}
}
// SetOption implements renderer.SetOptioner.
func (c *FootnoteConfig) SetOption(name renderer.OptionName, value interface{}) {
switch name {
case optFootnoteIDPrefixFunction:
c.IDPrefixFunction = value.(func(gast.Node) []byte)
case optFootnoteIDPrefix:
c.IDPrefix = value.([]byte)
case optFootnoteLinkTitle:
c.LinkTitle = value.([]byte)
case optFootnoteBacklinkTitle:
c.BacklinkTitle = value.([]byte)
case optFootnoteLinkClass:
c.LinkClass = value.([]byte)
case optFootnoteBacklinkClass:
c.BacklinkClass = value.([]byte)
case optFootnoteBacklinkHTML:
c.BacklinkHTML = value.([]byte)
default:
c.Config.SetOption(name, value)
}
}
type withFootnoteHTMLOptions struct {
value []html.Option
}
func (o *withFootnoteHTMLOptions) SetConfig(c *renderer.Config) {
if o.value != nil {
for _, v := range o.value {
v.(renderer.Option).SetConfig(c)
}
}
}
func (o *withFootnoteHTMLOptions) SetFootnoteOption(c *FootnoteConfig) {
if o.value != nil {
for _, v := range o.value {
v.SetHTMLOption(&c.Config)
}
}
}
// WithFootnoteHTMLOptions is functional option that wraps goldmark HTMLRenderer options.
func WithFootnoteHTMLOptions(opts ...html.Option) FootnoteOption {
return &withFootnoteHTMLOptions{opts}
}
const optFootnoteIDPrefix renderer.OptionName = "FootnoteIDPrefix"
type withFootnoteIDPrefix struct {
value []byte
}
func (o *withFootnoteIDPrefix) SetConfig(c *renderer.Config) {
c.Options[optFootnoteIDPrefix] = o.value
}
func (o *withFootnoteIDPrefix) SetFootnoteOption(c *FootnoteConfig) {
c.IDPrefix = o.value
}
// WithFootnoteIDPrefix is a functional option that is a prefix for the id attributes generated by footnotes.
func WithFootnoteIDPrefix(a []byte) FootnoteOption {
return &withFootnoteIDPrefix{a}
}
const optFootnoteIDPrefixFunction renderer.OptionName = "FootnoteIDPrefixFunction"
type withFootnoteIDPrefixFunction struct {
value func(gast.Node) []byte
}
func (o *withFootnoteIDPrefixFunction) SetConfig(c *renderer.Config) {
c.Options[optFootnoteIDPrefixFunction] = o.value
}
func (o *withFootnoteIDPrefixFunction) SetFootnoteOption(c *FootnoteConfig) {
c.IDPrefixFunction = o.value
}
// WithFootnoteIDPrefixFunction is a functional option that is a prefix for the id attributes generated by footnotes.
func WithFootnoteIDPrefixFunction(a func(gast.Node) []byte) FootnoteOption {
return &withFootnoteIDPrefixFunction{a}
}
const optFootnoteLinkTitle renderer.OptionName = "FootnoteLinkTitle"
type withFootnoteLinkTitle struct {
value []byte
}
func (o *withFootnoteLinkTitle) SetConfig(c *renderer.Config) {
c.Options[optFootnoteLinkTitle] = o.value
}
func (o *withFootnoteLinkTitle) SetFootnoteOption(c *FootnoteConfig) {
c.LinkTitle = o.value
}
// WithFootnoteLinkTitle is a functional option that is an optional title attribute for footnote links.
func WithFootnoteLinkTitle(a []byte) FootnoteOption {
return &withFootnoteLinkTitle{a}
}
const optFootnoteBacklinkTitle renderer.OptionName = "FootnoteBacklinkTitle"
type withFootnoteBacklinkTitle struct {
value []byte
}
func (o *withFootnoteBacklinkTitle) SetConfig(c *renderer.Config) {
c.Options[optFootnoteBacklinkTitle] = o.value
}
func (o *withFootnoteBacklinkTitle) SetFootnoteOption(c *FootnoteConfig) {
c.BacklinkTitle = o.value
}
// WithFootnoteBacklinkTitle is a functional option that is an optional title attribute for footnote backlinks.
func WithFootnoteBacklinkTitle(a []byte) FootnoteOption {
return &withFootnoteBacklinkTitle{a}
}
const optFootnoteLinkClass renderer.OptionName = "FootnoteLinkClass"
type withFootnoteLinkClass struct {
value []byte
}
func (o *withFootnoteLinkClass) SetConfig(c *renderer.Config) {
c.Options[optFootnoteLinkClass] = o.value
}
func (o *withFootnoteLinkClass) SetFootnoteOption(c *FootnoteConfig) {
c.LinkClass = o.value
}
// WithFootnoteLinkClass is a functional option that is a class for footnote links.
func WithFootnoteLinkClass(a []byte) FootnoteOption {
return &withFootnoteLinkClass{a}
}
const optFootnoteBacklinkClass renderer.OptionName = "FootnoteBacklinkClass"
type withFootnoteBacklinkClass struct {
value []byte
}
func (o *withFootnoteBacklinkClass) SetConfig(c *renderer.Config) {
c.Options[optFootnoteBacklinkClass] = o.value
}
func (o *withFootnoteBacklinkClass) SetFootnoteOption(c *FootnoteConfig) {
c.BacklinkClass = o.value
}
// WithFootnoteBacklinkClass is a functional option that is a class for footnote backlinks.
func WithFootnoteBacklinkClass(a []byte) FootnoteOption {
return &withFootnoteBacklinkClass{a}
}
const optFootnoteBacklinkHTML renderer.OptionName = "FootnoteBacklinkHTML"
type withFootnoteBacklinkHTML struct {
value []byte
}
func (o *withFootnoteBacklinkHTML) SetConfig(c *renderer.Config) {
c.Options[optFootnoteBacklinkHTML] = o.value
}
func (o *withFootnoteBacklinkHTML) SetFootnoteOption(c *FootnoteConfig) {
c.BacklinkHTML = o.value
}
// WithFootnoteBacklinkHTML is an HTML content for footnote backlinks.
func WithFootnoteBacklinkHTML(a []byte) FootnoteOption {
return &withFootnoteBacklinkHTML{a}
}
// FootnoteHTMLRenderer is a renderer.NodeRenderer implementation that // FootnoteHTMLRenderer is a renderer.NodeRenderer implementation that
// renders FootnoteLink nodes. // renders FootnoteLink nodes.
type FootnoteHTMLRenderer struct { type FootnoteHTMLRenderer struct {
html.Config FootnoteConfig
} }
// NewFootnoteHTMLRenderer returns a new FootnoteHTMLRenderer. // NewFootnoteHTMLRenderer returns a new FootnoteHTMLRenderer.
func NewFootnoteHTMLRenderer(opts ...html.Option) renderer.NodeRenderer { func NewFootnoteHTMLRenderer(opts ...FootnoteOption) renderer.NodeRenderer {
r := &FootnoteHTMLRenderer{ r := &FootnoteHTMLRenderer{
Config: html.NewConfig(), FootnoteConfig: NewFootnoteConfig(),
} }
for _, opt := range opts { for _, opt := range opts {
opt.SetHTMLOption(&r.Config) opt.SetFootnoteOption(&r.FootnoteConfig)
} }
return r return r
} }
@@ -234,7 +503,7 @@ func NewFootnoteHTMLRenderer(opts ...html.Option) renderer.NodeRenderer {
// RegisterFuncs implements renderer.NodeRenderer.RegisterFuncs. // RegisterFuncs implements renderer.NodeRenderer.RegisterFuncs.
func (r *FootnoteHTMLRenderer) RegisterFuncs(reg renderer.NodeRendererFuncRegisterer) { func (r *FootnoteHTMLRenderer) RegisterFuncs(reg renderer.NodeRendererFuncRegisterer) {
reg.Register(ast.KindFootnoteLink, r.renderFootnoteLink) reg.Register(ast.KindFootnoteLink, r.renderFootnoteLink)
reg.Register(ast.KindFootnoteBackLink, r.renderFootnoteBackLink) reg.Register(ast.KindFootnoteBacklink, r.renderFootnoteBacklink)
reg.Register(ast.KindFootnote, r.renderFootnote) reg.Register(ast.KindFootnote, r.renderFootnote)
reg.Register(ast.KindFootnoteList, r.renderFootnoteList) reg.Register(ast.KindFootnoteList, r.renderFootnoteList)
} }
@@ -243,25 +512,45 @@ func (r *FootnoteHTMLRenderer) renderFootnoteLink(w util.BufWriter, source []byt
if entering { if entering {
n := node.(*ast.FootnoteLink) n := node.(*ast.FootnoteLink)
is := strconv.Itoa(n.Index) is := strconv.Itoa(n.Index)
_, _ = w.WriteString(`<sup id="fnref:`) _, _ = w.WriteString(`<sup id="`)
_, _ = w.Write(r.idPrefix(node))
_, _ = w.WriteString(`fnref:`)
_, _ = w.WriteString(is) _, _ = w.WriteString(is)
_, _ = w.WriteString(`"><a href="#fn:`) _, _ = w.WriteString(`"><a href="#`)
_, _ = w.Write(r.idPrefix(node))
_, _ = w.WriteString(`fn:`)
_, _ = w.WriteString(is) _, _ = w.WriteString(is)
_, _ = w.WriteString(`" class="footnote-ref" role="doc-noteref">`) _, _ = w.WriteString(`" class="`)
_, _ = w.Write(applyFootnoteTemplate(r.FootnoteConfig.LinkClass,
n.Index, n.RefCount))
if len(r.FootnoteConfig.LinkTitle) > 0 {
_, _ = w.WriteString(`" title="`)
_, _ = w.Write(util.EscapeHTML(applyFootnoteTemplate(r.FootnoteConfig.LinkTitle, n.Index, n.RefCount)))
}
_, _ = w.WriteString(`" role="doc-noteref">`)
_, _ = w.WriteString(is) _, _ = w.WriteString(is)
_, _ = w.WriteString(`</a></sup>`) _, _ = w.WriteString(`</a></sup>`)
} }
return gast.WalkContinue, nil return gast.WalkContinue, nil
} }
func (r *FootnoteHTMLRenderer) renderFootnoteBackLink(w util.BufWriter, source []byte, node gast.Node, entering bool) (gast.WalkStatus, error) { func (r *FootnoteHTMLRenderer) renderFootnoteBacklink(w util.BufWriter, source []byte, node gast.Node, entering bool) (gast.WalkStatus, error) {
if entering { if entering {
n := node.(*ast.FootnoteBackLink) n := node.(*ast.FootnoteBacklink)
is := strconv.Itoa(n.Index) is := strconv.Itoa(n.Index)
_, _ = w.WriteString(` <a href="#fnref:`) _, _ = w.WriteString(` <a href="#`)
_, _ = w.Write(r.idPrefix(node))
_, _ = w.WriteString(`fnref:`)
_, _ = w.WriteString(is) _, _ = w.WriteString(is)
_, _ = w.WriteString(`" class="footnote-backref" role="doc-backlink">`) _, _ = w.WriteString(`" class="`)
_, _ = w.WriteString("&#x21a9;&#xfe0e;") _, _ = w.Write(applyFootnoteTemplate(r.FootnoteConfig.BacklinkClass, n.Index, n.RefCount))
if len(r.FootnoteConfig.BacklinkTitle) > 0 {
_, _ = w.WriteString(`" title="`)
_, _ = w.Write(util.EscapeHTML(applyFootnoteTemplate(r.FootnoteConfig.BacklinkTitle, n.Index, n.RefCount)))
}
_, _ = w.WriteString(`" role="doc-backlink">`)
_, _ = w.Write(applyFootnoteTemplate(r.FootnoteConfig.BacklinkHTML, n.Index, n.RefCount))
_, _ = w.WriteString(`</a>`) _, _ = w.WriteString(`</a>`)
} }
return gast.WalkContinue, nil return gast.WalkContinue, nil
@@ -271,7 +560,9 @@ func (r *FootnoteHTMLRenderer) renderFootnote(w util.BufWriter, source []byte, n
n := node.(*ast.Footnote) n := node.(*ast.Footnote)
is := strconv.Itoa(n.Index) is := strconv.Itoa(n.Index)
if entering { if entering {
_, _ = w.WriteString(`<li id="fn:`) _, _ = w.WriteString(`<li id="`)
_, _ = w.Write(r.idPrefix(node))
_, _ = w.WriteString(`fn:`)
_, _ = w.WriteString(is) _, _ = w.WriteString(is)
_, _ = w.WriteString(`" role="doc-endnote"`) _, _ = w.WriteString(`" role="doc-endnote"`)
if node.Attributes() != nil { if node.Attributes() != nil {
@@ -312,11 +603,54 @@ func (r *FootnoteHTMLRenderer) renderFootnoteList(w util.BufWriter, source []byt
return gast.WalkContinue, nil return gast.WalkContinue, nil
} }
func (r *FootnoteHTMLRenderer) idPrefix(node gast.Node) []byte {
if r.FootnoteConfig.IDPrefix != nil {
return r.FootnoteConfig.IDPrefix
}
if r.FootnoteConfig.IDPrefixFunction != nil {
return r.FootnoteConfig.IDPrefixFunction(node)
}
return []byte("")
}
func applyFootnoteTemplate(b []byte, index, refCount int) []byte {
fast := true
for i, c := range b {
if i != 0 {
if b[i-1] == '^' && c == '^' {
fast = false
break
}
if b[i-1] == '%' && c == '%' {
fast = false
break
}
}
}
if fast {
return b
}
is := []byte(strconv.Itoa(index))
rs := []byte(strconv.Itoa(refCount))
ret := bytes.Replace(b, []byte("^^"), is, -1)
return bytes.Replace(ret, []byte("%%"), rs, -1)
}
type footnote struct { type footnote struct {
options []FootnoteOption
} }
// Footnote is an extension that allow you to use PHP Markdown Extra Footnotes. // Footnote is an extension that allow you to use PHP Markdown Extra Footnotes.
var Footnote = &footnote{} var Footnote = &footnote{
options: []FootnoteOption{},
}
// NewFootnote returns a new extension with given options.
func NewFootnote(opts ...FootnoteOption) goldmark.Extender {
return &footnote{
options: opts,
}
}
func (e *footnote) Extend(m goldmark.Markdown) { func (e *footnote) Extend(m goldmark.Markdown) {
m.Parser().AddOptions( m.Parser().AddOptions(
@@ -331,6 +665,6 @@ func (e *footnote) Extend(m goldmark.Markdown) {
), ),
) )
m.Renderer().AddOptions(renderer.WithNodeRenderers( m.Renderer().AddOptions(renderer.WithNodeRenderers(
util.Prioritized(NewFootnoteHTMLRenderer(), 500), util.Prioritized(NewFootnoteHTMLRenderer(e.options...), 500),
)) ))
} }

View File

@@ -11,9 +11,9 @@ import (
"github.com/yuin/goldmark/util" "github.com/yuin/goldmark/util"
) )
var wwwURLRegxp = regexp.MustCompile(`^www\.[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]+(?:(?:/|[#?])[-a-zA-Z0-9@:%_\+.~#!?&//=\(\);,'">\^{}\[\]` + "`" + `]*)?`) var wwwURLRegxp = regexp.MustCompile(`^www\.[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-z]+(?:[/#?][-a-zA-Z0-9@:%_\+.~#!?&/=\(\);,'">\^{}\[\]` + "`" + `]*)?`)
var urlRegexp = regexp.MustCompile(`^(?:http|https|ftp):\/\/(?:www\.)?[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]+(?:(?:/|[#?])[-a-zA-Z0-9@:%_+.~#$!?&//=\(\);,'">\^{}\[\]` + "`" + `]*)?`) var urlRegexp = regexp.MustCompile(`^(?:http|https|ftp)://[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-z]+(?::\d+)?(?:[/#?][-a-zA-Z0-9@:%_+.~#$!?&/=\(\);,'">\^{}\[\]` + "`" + `]*)?`)
// An LinkifyConfig struct is a data structure that holds configuration of the // An LinkifyConfig struct is a data structure that holds configuration of the
// Linkify extension. // Linkify extension.
@@ -24,10 +24,12 @@ type LinkifyConfig struct {
EmailRegexp *regexp.Regexp EmailRegexp *regexp.Regexp
} }
const optLinkifyAllowedProtocols parser.OptionName = "LinkifyAllowedProtocols" const (
const optLinkifyURLRegexp parser.OptionName = "LinkifyURLRegexp" optLinkifyAllowedProtocols parser.OptionName = "LinkifyAllowedProtocols"
const optLinkifyWWWRegexp parser.OptionName = "LinkifyWWWRegexp" optLinkifyURLRegexp parser.OptionName = "LinkifyURLRegexp"
const optLinkifyEmailRegexp parser.OptionName = "LinkifyEmailRegexp" optLinkifyWWWRegexp parser.OptionName = "LinkifyWWWRegexp"
optLinkifyEmailRegexp parser.OptionName = "LinkifyEmailRegexp"
)
// SetOption implements SetOptioner. // SetOption implements SetOptioner.
func (c *LinkifyConfig) SetOption(name parser.OptionName, value interface{}) { func (c *LinkifyConfig) SetOption(name parser.OptionName, value interface{}) {
@@ -156,10 +158,12 @@ func (s *linkifyParser) Trigger() []byte {
return []byte{' ', '*', '_', '~', '('} return []byte{' ', '*', '_', '~', '('}
} }
var protoHTTP = []byte("http:") var (
var protoHTTPS = []byte("https:") protoHTTP = []byte("http:")
var protoFTP = []byte("ftp:") protoHTTPS = []byte("https:")
var domainWWW = []byte("www.") protoFTP = []byte("ftp:")
domainWWW = []byte("www.")
)
func (s *linkifyParser) Parse(parent ast.Node, block text.Reader, pc parser.Context) ast.Node { func (s *linkifyParser) Parse(parent ast.Node, block text.Reader, pc parser.Context) ast.Node {
if pc.IsInLinkLabel() { if pc.IsInLinkLabel() {

View File

@@ -15,6 +15,14 @@ import (
"github.com/yuin/goldmark/util" "github.com/yuin/goldmark/util"
) )
var escapedPipeCellListKey = parser.NewContextKey()
type escapedPipeCell struct {
Cell *ast.TableCell
Pos []int
Transformed bool
}
// TableCellAlignMethod indicates how are table cells aligned in HTML format.indicates how are table cells aligned in HTML format. // TableCellAlignMethod indicates how are table cells aligned in HTML format.indicates how are table cells aligned in HTML format.
type TableCellAlignMethod int type TableCellAlignMethod int
@@ -148,7 +156,7 @@ func (b *tableParagraphTransformer) Transform(node *gast.Paragraph, reader text.
if alignments == nil { if alignments == nil {
continue continue
} }
header := b.parseRow(lines.At(i-1), alignments, true, reader) header := b.parseRow(lines.At(i-1), alignments, true, reader, pc)
if header == nil || len(alignments) != header.ChildCount() { if header == nil || len(alignments) != header.ChildCount() {
return return
} }
@@ -156,7 +164,7 @@ func (b *tableParagraphTransformer) Transform(node *gast.Paragraph, reader text.
table.Alignments = alignments table.Alignments = alignments
table.AppendChild(table, ast.NewTableHeader(header)) table.AppendChild(table, ast.NewTableHeader(header))
for j := i + 1; j < lines.Len(); j++ { for j := i + 1; j < lines.Len(); j++ {
table.AppendChild(table, b.parseRow(lines.At(j), alignments, false, reader)) table.AppendChild(table, b.parseRow(lines.At(j), alignments, false, reader, pc))
} }
node.Lines().SetSliced(0, i-1) node.Lines().SetSliced(0, i-1)
node.Parent().InsertAfter(node.Parent(), node, table) node.Parent().InsertAfter(node.Parent(), node, table)
@@ -170,7 +178,7 @@ func (b *tableParagraphTransformer) Transform(node *gast.Paragraph, reader text.
} }
} }
func (b *tableParagraphTransformer) parseRow(segment text.Segment, alignments []ast.Alignment, isHeader bool, reader text.Reader) *ast.TableRow { func (b *tableParagraphTransformer) parseRow(segment text.Segment, alignments []ast.Alignment, isHeader bool, reader text.Reader, pc parser.Context) *ast.TableRow {
source := reader.Source() source := reader.Source()
line := segment.Value(source) line := segment.Value(source)
pos := 0 pos := 0
@@ -194,18 +202,39 @@ func (b *tableParagraphTransformer) parseRow(segment text.Segment, alignments []
} else { } else {
alignment = alignments[i] alignment = alignments[i]
} }
closure := util.FindClosure(line[pos:], byte(0), '|', true, false)
if closure < 0 { var escapedCell *escapedPipeCell
closure = len(line[pos:])
}
node := ast.NewTableCell() node := ast.NewTableCell()
seg := text.NewSegment(segment.Start+pos, segment.Start+pos+closure) node.Alignment = alignment
hasBacktick := false
closure := pos
for ; closure < limit; closure++ {
if line[closure] == '`' {
hasBacktick = true
}
if line[closure] == '|' {
if closure == 0 || line[closure-1] != '\\' {
break
} else if hasBacktick {
if escapedCell == nil {
escapedCell = &escapedPipeCell{node, []int{}, false}
escapedList := pc.ComputeIfAbsent(escapedPipeCellListKey,
func() interface{} {
return []*escapedPipeCell{}
}).([]*escapedPipeCell)
escapedList = append(escapedList, escapedCell)
pc.Set(escapedPipeCellListKey, escapedList)
}
escapedCell.Pos = append(escapedCell.Pos, segment.Start+closure-1)
}
}
}
seg := text.NewSegment(segment.Start+pos, segment.Start+closure)
seg = seg.TrimLeftSpace(source) seg = seg.TrimLeftSpace(source)
seg = seg.TrimRightSpace(source) seg = seg.TrimRightSpace(source)
node.Lines().Append(seg) node.Lines().Append(seg)
node.Alignment = alignment
row.AppendChild(row, node) row.AppendChild(row, node)
pos += closure + 1 pos = closure + 1
} }
for ; i < len(alignments); i++ { for ; i < len(alignments); i++ {
row.AppendChild(row, ast.NewTableCell()) row.AppendChild(row, ast.NewTableCell())
@@ -243,6 +272,61 @@ func (b *tableParagraphTransformer) parseDelimiter(segment text.Segment, reader
return alignments return alignments
} }
type tableASTTransformer struct {
}
var defaultTableASTTransformer = &tableASTTransformer{}
// NewTableASTTransformer returns a parser.ASTTransformer for tables.
func NewTableASTTransformer() parser.ASTTransformer {
return defaultTableASTTransformer
}
func (a *tableASTTransformer) Transform(node *gast.Document, reader text.Reader, pc parser.Context) {
lst := pc.Get(escapedPipeCellListKey)
if lst == nil {
return
}
pc.Set(escapedPipeCellListKey, nil)
for _, v := range lst.([]*escapedPipeCell) {
if v.Transformed {
continue
}
_ = gast.Walk(v.Cell, func(n gast.Node, entering bool) (gast.WalkStatus, error) {
if !entering || n.Kind() != gast.KindCodeSpan {
return gast.WalkContinue, nil
}
for c := n.FirstChild(); c != nil; {
next := c.NextSibling()
if c.Kind() != gast.KindText {
c = next
continue
}
parent := c.Parent()
ts := &c.(*gast.Text).Segment
n := c
for _, v := range lst.([]*escapedPipeCell) {
for _, pos := range v.Pos {
if ts.Start <= pos && pos < ts.Stop {
segment := n.(*gast.Text).Segment
n1 := gast.NewRawTextSegment(segment.WithStop(pos))
n2 := gast.NewRawTextSegment(segment.WithStart(pos + 1))
parent.InsertAfter(parent, n, n1)
parent.InsertAfter(parent, n1, n2)
parent.RemoveChild(parent, n)
n = n2
v.Transformed = true
}
}
}
c = next
}
return gast.WalkContinue, nil
})
}
}
// TableHTMLRenderer is a renderer.NodeRenderer implementation that // TableHTMLRenderer is a renderer.NodeRenderer implementation that
// renders Table nodes. // renders Table nodes.
type TableHTMLRenderer struct { type TableHTMLRenderer struct {
@@ -419,7 +503,7 @@ func (r *TableHTMLRenderer) renderTableCell(w util.BufWriter, source []byte, nod
cob.AppendByte(';') cob.AppendByte(';')
} }
style := fmt.Sprintf("text-align:%s", n.Alignment.String()) style := fmt.Sprintf("text-align:%s", n.Alignment.String())
cob.Append(util.StringToReadOnlyBytes(style)) cob.AppendString(style)
n.SetAttributeString("style", cob.Bytes()) n.SetAttributeString("style", cob.Bytes())
} }
} }
@@ -454,9 +538,14 @@ func NewTable(opts ...TableOption) goldmark.Extender {
} }
func (e *table) Extend(m goldmark.Markdown) { func (e *table) Extend(m goldmark.Markdown) {
m.Parser().AddOptions(parser.WithParagraphTransformers( m.Parser().AddOptions(
util.Prioritized(NewTableParagraphTransformer(), 200), parser.WithParagraphTransformers(
)) util.Prioritized(NewTableParagraphTransformer(), 200),
),
parser.WithASTTransformers(
util.Prioritized(defaultTableASTTransformer, 0),
),
)
m.Renderer().AddOptions(renderer.WithNodeRenderers( m.Renderer().AddOptions(renderer.WithNodeRenderers(
util.Prioritized(NewTableHTMLRenderer(e.options...), 500), util.Prioritized(NewTableHTMLRenderer(e.options...), 500),
)) ))

View File

@@ -1,3 +1,3 @@
module github.com/yuin/goldmark module github.com/yuin/goldmark
go 1.13 go 1.15

View File

@@ -49,6 +49,12 @@ func (b *codeBlockParser) Continue(node ast.Node, reader text.Reader, pc Context
} }
reader.AdvanceAndSetPadding(pos, padding) reader.AdvanceAndSetPadding(pos, padding)
_, segment = reader.PeekLine() _, segment = reader.PeekLine()
// if code block line starts with a tab, keep a tab as it is.
if segment.Padding != 0 {
preserveLeadingTabInCodeBlock(&segment, reader)
}
node.Lines().Append(segment) node.Lines().Append(segment)
reader.Advance(segment.Len() - 1) reader.Advance(segment.Len() - 1)
return Continue | NoChildren return Continue | NoChildren
@@ -77,3 +83,14 @@ func (b *codeBlockParser) CanInterruptParagraph() bool {
func (b *codeBlockParser) CanAcceptIndentedLine() bool { func (b *codeBlockParser) CanAcceptIndentedLine() bool {
return true return true
} }
func preserveLeadingTabInCodeBlock(segment *text.Segment, reader text.Reader) {
offsetWithPadding := reader.LineOffset()
sl, ss := reader.Position()
reader.SetPosition(sl, text.NewSegment(ss.Start-1, ss.Stop))
if offsetWithPadding == reader.LineOffset() {
segment.Padding = 0
segment.Start--
}
reader.SetPosition(sl, ss)
}

View File

@@ -71,6 +71,10 @@ func (b *fencedCodeBlockParser) Open(parent ast.Node, reader text.Reader, pc Con
func (b *fencedCodeBlockParser) Continue(node ast.Node, reader text.Reader, pc Context) State { func (b *fencedCodeBlockParser) Continue(node ast.Node, reader text.Reader, pc Context) State {
line, segment := reader.PeekLine() line, segment := reader.PeekLine()
fdata := pc.Get(fencedCodeBlockInfoKey).(*fenceData) fdata := pc.Get(fencedCodeBlockInfoKey).(*fenceData)
// if code block line starts with a tab, keep a tab as it is.
if segment.Padding != 0 {
preserveLeadingTabInCodeBlock(&segment, reader)
}
w, pos := util.IndentWidth(line, reader.LineOffset()) w, pos := util.IndentWidth(line, reader.LineOffset())
if w < 4 { if w < 4 {
i := pos i := pos

View File

@@ -2,7 +2,6 @@ package parser
import ( import (
"fmt" "fmt"
"regexp"
"strings" "strings"
"github.com/yuin/goldmark/ast" "github.com/yuin/goldmark/ast"
@@ -113,8 +112,6 @@ func (s *linkParser) Trigger() []byte {
return []byte{'!', '[', ']'} return []byte{'!', '[', ']'}
} }
var linkDestinationRegexp = regexp.MustCompile(`\s*([^\s].+)`)
var linkTitleRegexp = regexp.MustCompile(`\s+(\)|["'\(].+)`)
var linkBottom = NewContextKey() var linkBottom = NewContextKey()
func (s *linkParser) Parse(parent ast.Node, block text.Reader, pc Context) ast.Node { func (s *linkParser) Parse(parent ast.Node, block text.Reader, pc Context) ast.Node {
@@ -293,20 +290,17 @@ func (s *linkParser) parseLink(parent ast.Node, last *linkLabelState, block text
func parseLinkDestination(block text.Reader) ([]byte, bool) { func parseLinkDestination(block text.Reader) ([]byte, bool) {
block.SkipSpaces() block.SkipSpaces()
line, _ := block.PeekLine() line, _ := block.PeekLine()
buf := []byte{}
if block.Peek() == '<' { if block.Peek() == '<' {
i := 1 i := 1
for i < len(line) { for i < len(line) {
c := line[i] c := line[i]
if c == '\\' && i < len(line)-1 && util.IsPunct(line[i+1]) { if c == '\\' && i < len(line)-1 && util.IsPunct(line[i+1]) {
buf = append(buf, '\\', line[i+1])
i += 2 i += 2
continue continue
} else if c == '>' { } else if c == '>' {
block.Advance(i + 1) block.Advance(i + 1)
return line[1:i], true return line[1:i], true
} }
buf = append(buf, c)
i++ i++
} }
return nil, false return nil, false
@@ -316,7 +310,6 @@ func parseLinkDestination(block text.Reader) ([]byte, bool) {
for i < len(line) { for i < len(line) {
c := line[i] c := line[i]
if c == '\\' && i < len(line)-1 && util.IsPunct(line[i+1]) { if c == '\\' && i < len(line)-1 && util.IsPunct(line[i+1]) {
buf = append(buf, '\\', line[i+1])
i += 2 i += 2
continue continue
} else if c == '(' { } else if c == '(' {
@@ -329,7 +322,6 @@ func parseLinkDestination(block text.Reader) ([]byte, bool) {
} else if util.IsSpace(c) { } else if util.IsSpace(c) {
break break
} }
buf = append(buf, c)
i++ i++
} }
block.Advance(i) block.Advance(i)

View File

@@ -138,6 +138,9 @@ type Context interface {
// Get returns a value associated with the given key. // Get returns a value associated with the given key.
Get(ContextKey) interface{} Get(ContextKey) interface{}
// ComputeIfAbsent computes a value if a value associated with the given key is absent and returns the value.
ComputeIfAbsent(ContextKey, func() interface{}) interface{}
// Set sets the given value to the context. // Set sets the given value to the context.
Set(ContextKey, interface{}) Set(ContextKey, interface{})
@@ -252,6 +255,15 @@ func (p *parseContext) Get(key ContextKey) interface{} {
return p.store[key] return p.store[key]
} }
func (p *parseContext) ComputeIfAbsent(key ContextKey, f func() interface{}) interface{} {
v := p.store[key]
if v == nil {
v = f()
p.store[key] = v
}
return v
}
func (p *parseContext) Set(key ContextKey, value interface{}) { func (p *parseContext) Set(key ContextKey, value interface{}) {
p.store[key] = value p.store[key] = value
} }

View File

@@ -2,10 +2,11 @@ package parser
import ( import (
"bytes" "bytes"
"regexp"
"github.com/yuin/goldmark/ast" "github.com/yuin/goldmark/ast"
"github.com/yuin/goldmark/text" "github.com/yuin/goldmark/text"
"github.com/yuin/goldmark/util" "github.com/yuin/goldmark/util"
"regexp"
) )
type rawHTMLParser struct { type rawHTMLParser struct {
@@ -67,8 +68,6 @@ func (s *rawHTMLParser) parseSingleLineRegexp(reg *regexp.Regexp, block text.Rea
return node return node
} }
var dummyMatch = [][]byte{}
func (s *rawHTMLParser) parseMultiLineRegexp(reg *regexp.Regexp, block text.Reader, pc Context) ast.Node { func (s *rawHTMLParser) parseMultiLineRegexp(reg *regexp.Regexp, block text.Reader, pc Context) ast.Node {
sline, ssegment := block.Position() sline, ssegment := block.Position()
if block.Match(reg) { if block.Match(reg) {
@@ -102,7 +101,3 @@ func (s *rawHTMLParser) parseMultiLineRegexp(reg *regexp.Regexp, block text.Read
} }
return nil return nil
} }
func (s *rawHTMLParser) CloseBlock(parent ast.Node, pc Context) {
// nothing to do
}

View File

@@ -37,6 +37,12 @@ func (b *CopyOnWriteBuffer) Write(value []byte) {
b.buffer = append(b.buffer, value...) b.buffer = append(b.buffer, value...)
} }
// WriteString writes given string to the buffer.
// WriteString allocate new buffer and clears it at the first time.
func (b *CopyOnWriteBuffer) WriteString(value string) {
b.Write(StringToReadOnlyBytes(value))
}
// Append appends given bytes to the buffer. // Append appends given bytes to the buffer.
// Append copy buffer at the first time. // Append copy buffer at the first time.
func (b *CopyOnWriteBuffer) Append(value []byte) { func (b *CopyOnWriteBuffer) Append(value []byte) {
@@ -49,6 +55,12 @@ func (b *CopyOnWriteBuffer) Append(value []byte) {
b.buffer = append(b.buffer, value...) b.buffer = append(b.buffer, value...)
} }
// AppendString appends given string to the buffer.
// AppendString copy buffer at the first time.
func (b *CopyOnWriteBuffer) AppendString(value string) {
b.Append(StringToReadOnlyBytes(value))
}
// WriteByte writes the given byte to the buffer. // WriteByte writes the given byte to the buffer.
// WriteByte allocate new buffer and clears it at the first time. // WriteByte allocate new buffer and clears it at the first time.
func (b *CopyOnWriteBuffer) WriteByte(c byte) { func (b *CopyOnWriteBuffer) WriteByte(c byte) {
@@ -804,7 +816,7 @@ func IsPunct(c byte) bool {
return punctTable[c] == 1 return punctTable[c] == 1
} }
// IsPunct returns true if the given rune is a punctuation, otherwise false. // IsPunctRune returns true if the given rune is a punctuation, otherwise false.
func IsPunctRune(r rune) bool { func IsPunctRune(r rune) bool {
return int32(r) <= 256 && IsPunct(byte(r)) || unicode.IsPunct(r) return int32(r) <= 256 && IsPunct(byte(r)) || unicode.IsPunct(r)
} }
@@ -814,7 +826,7 @@ func IsSpace(c byte) bool {
return spaceTable[c] == 1 return spaceTable[c] == 1
} }
// IsSpace returns true if the given rune is a space, otherwise false. // IsSpaceRune returns true if the given rune is a space, otherwise false.
func IsSpaceRune(r rune) bool { func IsSpaceRune(r rune) bool {
return int32(r) <= 256 && IsSpace(byte(r)) || unicode.IsSpace(r) return int32(r) <= 256 && IsSpace(byte(r)) || unicode.IsSpace(r)
} }

2
vendor/modules.txt vendored
View File

@@ -745,7 +745,7 @@ github.com/xi2/xz
# github.com/yohcop/openid-go v1.0.0 # github.com/yohcop/openid-go v1.0.0
## explicit ## explicit
github.com/yohcop/openid-go github.com/yohcop/openid-go
# github.com/yuin/goldmark v1.2.1 # github.com/yuin/goldmark v1.3.3
## explicit ## explicit
github.com/yuin/goldmark github.com/yuin/goldmark
github.com/yuin/goldmark/ast github.com/yuin/goldmark/ast

View File

@@ -3592,18 +3592,21 @@ function initIssueList() {
fullTextSearch: true fullTextSearch: true
}); });
function excludeLabel (item) {
const href = $(item).attr('href');
const id = $(item).data('label-id');
const regStr = `labels=((?:-?[0-9]+%2c)*)(${id})((?:%2c-?[0-9]+)*)&`;
const newStr = 'labels=$1-$2$3&';
window.location = href.replace(new RegExp(regStr), newStr);
}
$('.menu a.label-filter-item').each(function () { $('.menu a.label-filter-item').each(function () {
$(this).on('click', function (e) { $(this).on('click', function (e) {
if (e.altKey) { if (e.altKey) {
e.preventDefault(); e.preventDefault();
excludeLabel(this);
const href = $(this).attr('href');
const id = $(this).data('label-id');
const regStr = `labels=(-?[0-9]+%2c)*(${id})(%2c-?[0-9]+)*&`;
const newStr = 'labels=$1-$2$3&';
window.location = href.replace(new RegExp(regStr), newStr);
} }
}); });
}); });
@@ -3611,17 +3614,8 @@ function initIssueList() {
$('.menu .ui.dropdown.label-filter').on('keydown', (e) => { $('.menu .ui.dropdown.label-filter').on('keydown', (e) => {
if (e.altKey && e.keyCode === 13) { if (e.altKey && e.keyCode === 13) {
const selectedItems = $('.menu .ui.dropdown.label-filter .menu .item.selected'); const selectedItems = $('.menu .ui.dropdown.label-filter .menu .item.selected');
if (selectedItems.length > 0) { if (selectedItems.length > 0) {
const item = $(selectedItems[0]); excludeLabel($(selectedItems[0]));
const href = item.attr('href');
const id = item.data('label-id');
const regStr = `labels=(-?[0-9]+%2c)*(${id})(%2c-?[0-9]+)*&`;
const newStr = 'labels=$1-$2$3&';
window.location = href.replace(new RegExp(regStr), newStr);
} }
} }
}); });

View File

@@ -5,7 +5,7 @@ const headingSelector = '.markdown h1, .markdown h2, .markdown h3, .markdown h4,
function scrollToAnchor() { function scrollToAnchor() {
if (document.querySelector(':target')) return; if (document.querySelector(':target')) return;
if (!window.location.hash || window.location.hash.length <= 1) return; if (!window.location.hash || window.location.hash.length <= 1) return;
const id = window.location.hash.substring(1); const id = decodeURIComponent(window.location.hash.substring(1));
const el = document.getElementById(`user-content-${id}`); const el = document.getElementById(`user-content-${id}`);
if (el) { if (el) {
el.scrollIntoView(); el.scrollIntoView();

View File

@@ -1320,6 +1320,7 @@
padding-top: 5px; padding-top: 5px;
padding-right: 10px; padding-right: 10px;
color: #000000; color: #000000;
font-size: 1.5rem;
&:hover { &:hover {
color: #4078c0; color: #4078c0;