Compare commits

...

22 Commits

Author SHA1 Message Date
6543
6dfa92bb1c Changelog v1.13.6 (#15129) 2021-03-23 15:44:50 -04:00
6543
151bedab52 Fix bug on avatar middleware (#15125)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-03-23 18:45:06 +00:00
zeripath
6198403fbc Fix another clusterfuzz identified issue (#15096) (#15114)
Backport #15096

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-03-22 16:27:21 -04:00
a1012112796
a6290f603f fix #15104 (#15106)
Signed-off-by: a1012112796 <1012112796@qq.com>
2021-03-22 15:15:44 -04:00
silverwind
2f09e5775f Fix markdown rendering in milestone content (#15056) (#15092)
- Add missing markdown class for rendered markdown.
- Increase font size of milestone name in list.

Fixes: https://github.com/go-gitea/gitea/issues/15046
2021-03-21 18:03:52 +01:00
zeripath
b0819efaea Place wrapper around comment as diff to catch panics (#15085) (#15086)
* Place wrapper around comment as diff to prevent panics

* propagate the panic up

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-21 16:16:07 +01:00
6543
d7a3bcdd70 Changelog v1.13.5 (#15084) 2021-03-21 15:05:21 +01:00
zeripath
7a85e228d8 Update to goldmark 1.3.3 (#15059) (#15061)
Backport #15059

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-20 10:31:28 +00:00
6543
a461d90415 Fix bug when upload on web (#15042) (#15055)
* Fix bug when upload on web

* move into own function

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
2021-03-20 09:37:53 +08:00
6543
70e4134130 Delete Labels & IssueLabels on Repo Delete too (#15039) (#15051)
* Doctor: find IssueLabels without existing label

* Repo Delete: delete labels & issue_labels too
2021-03-19 22:13:39 +01:00
zeripath
909f2be99d Fix postgres ID sequences broken by recreate-table (#15015) (#15029)
Backport #15015

Unfortunately there is a subtle problem with recreatetable on postgres which
leads to the sequences not being renamed and not being left at 0.

Fix #14725

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-19 04:23:58 +01:00
6543
645c0d8abd another clusterfuzz spotted issue (#15032) (#15034)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-03-19 00:21:33 +02:00
zeripath
8c461eb261 Fix several render issues (#14986) (#15013)
Backport #14986

* Fix an issue with panics related to attributes
* Wrap goldmark render in a recovery function
* Reduce memory use in render emoji
* Use a pipe for rendering goldmark - still needs more work and a limiter

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-03-17 10:58:58 +02:00
Norwin
fff66eb016 API: fix set milestone on PR creation (#14981) (#15001)
* API: fix set milestone on PR creation

pr creation via API failed with 404, because we searched
for milestoneID 0, due to uninitialized var usage D:

* add tests

Co-authored-by: 6543 <6543@obermui.de>

Co-authored-by: 6543 <6543@obermui.de>
2021-03-15 11:01:04 -04:00
zeripath
c965ed6529 Make sure sibling images get a link too (#14979) (#14995)
Backport #14979

Due a problem with the ast.Walker in the our transformer in goldmark
an image with a sibling image will not be transformed to gain a parent
link. This PR fixes this.

Fix #12925

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-15 12:34:56 +08:00
zeripath
71a2adbf10 Fix Anchor jumping with escaped query components (#14969) (#14977)
Backport #14969

Fix #14968

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-13 09:54:53 +00:00
Norwin
3231b70043 check if original author is set (#14972)
Co-authored-by: 6543 <6543@obermui.de>
2021-03-13 11:05:56 +08:00
Norwin
e3c44923d7 fix release mail html template (#14976)
was missing an </a>
2021-03-12 20:39:05 +00:00
zeripath
3e7dccdf47 Fix excluding more than two labels on issues list (#14962) (#14973)
Backport #14962

Fix #14840

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Norwin Roosen <git@nroo.de>
Co-authored-by: jaqra <48099350+jaqra@users.noreply.github.com>

Co-authored-by: Norwin Roosen <git@nroo.de>
Co-authored-by: jaqra <48099350+jaqra@users.noreply.github.com>
2021-03-12 18:12:14 +01:00
6543
33c2c49627 Prevent panic when editing forked repos by API (#14960) (#14963)
When editing forked repos using the API the BaseRepository needs to loaded
in order to check its visibility otherwise there will be NPE panic.

Fix #14956

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-03-12 08:54:18 +08:00
fnetX (aka fralix)
05ac72cf33 Add "captcha" to list of reserved usernames (#14930)
Signed-off-by: Otto Richter <git@fralix.ovh>
2021-03-08 17:50:13 +01:00
zeripath
906ecfd173 Re-enable import local paths after reversion from #13610 (#14925) (#14927)
Backport #14925

PR #13610 unfortunately disabled importing repositories from local paths.
This PR restores this functionality.

Fix #14700

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-03-08 14:50:57 +01:00
50 changed files with 1356 additions and 194 deletions

View File

@@ -4,6 +4,37 @@ This changelog goes through all the changes that have been made in each release
without substantial changes to our git log; to see the highlights of what has
been added to each release, please refer to the [blog](https://blog.gitea.io).
## [1.13.6](https://github.com/go-gitea/gitea/releases/tag/v1.13.6) - 2021-03-23
* SECURITY
* Fix bug on avatar middleware (#15124) (#15125)
* Fix another clusterfuzz identified issue (#15096) (#15114)
* API
* Fix nil exeption for get pull reviews API #15104 (#15106)
* BUGFIXES
* Fix markdown rendering in milestone content (#15056) (#15092)
## [1.13.5](https://github.com/go-gitea/gitea/releases/tag/v1.13.5) - 2021-03-21
* SECURITY
* Update to goldmark 1.3.3 (#15059) (#15061)
* API
* Fix set milestone on PR creation (#14981) (#15001)
* Prevent panic when editing forked repos by API (#14960) (#14963)
* BUGFIXES
* Fix bug when upload on web (#15042) (#15055)
* Delete Labels & IssueLabels on Repo Delete too (#15039) (#15051)
* another clusterfuzz spotted issue (#15032) (#15034)
* Fix postgres ID sequences broken by recreate-table (#15015) (#15029)
* Fix several render issues (#14986) (#15013)
* Make sure sibling images get a link too (#14979) (#14995)
* Fix Anchor jumping with escaped query components (#14969) (#14977)
* fix release mail html template (#14976)
* Fix excluding more than two labels on issues list (#14962) (#14973)
* don't mark each comment poster as OP (#14971) (#14972)
* Add "captcha" to list of reserved usernames (#14930)
* Re-enable import local paths after reversion from #13610 (#14925) (#14927)
## [1.13.4](https://github.com/go-gitea/gitea/releases/tag/v1.13.4) - 2021-03-07
* SECURITY
@@ -207,7 +238,7 @@ been added to each release, please refer to the [blog](https://blog.gitea.io).
* Fix scrolling to resolved comment anchors (#13343) (#13371)
* Storage configuration support `[storage]` (#13314) (#13379)
* When creating line diffs do not split within an html entity (#13357) (#13375) (#13425) (#13427)
* Fix reactions on code comments (#13390) (#13401)
* Fix reactions on code comments (#13390) (#13401)
* Add missing full names when DEFAULT_SHOW_FULL_NAME is enabled (#13424)
* Replies to outdated code comments should also be outdated (#13217) (#13433)
* Fix panic bug in handling multiple references in commit (#13486) (#13487)

View File

@@ -606,6 +606,22 @@ func runDoctorCheckDBConsistency(ctx *cli.Context) ([]string, error) {
}
}
// find IssueLabels without existing label
count, err = models.CountOrphanedIssueLabels()
if err != nil {
return nil, err
}
if count > 0 {
if ctx.Bool("fix") {
if err = models.DeleteOrphanedIssueLabels(); err != nil {
return nil, err
}
results = append(results, fmt.Sprintf("%d issue_labels without existing label deleted", count))
} else {
results = append(results, fmt.Sprintf("%d issue_labels without existing label", count))
}
}
//find issues without existing repository
count, err = models.CountOrphanedIssues()
if err != nil {
@@ -670,6 +686,23 @@ func runDoctorCheckDBConsistency(ctx *cli.Context) ([]string, error) {
}
}
if setting.Database.UsePostgreSQL {
count, err = models.CountBadSequences()
if err != nil {
return nil, err
}
if count > 0 {
if ctx.Bool("fix") {
err := models.FixBadSequences()
if err != nil {
return nil, err
}
results = append(results, fmt.Sprintf("%d sequences updated", count))
} else {
results = append(results, fmt.Sprintf("%d sequences with incorrect values", count))
}
}
}
//ToDo: function to recalc all counters
return results, nil

2
go.mod
View File

@@ -99,7 +99,7 @@ require (
github.com/urfave/cli v1.20.0
github.com/xanzy/go-gitlab v0.37.0
github.com/yohcop/openid-go v1.0.0
github.com/yuin/goldmark v1.2.1
github.com/yuin/goldmark v1.3.3
github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691
github.com/yuin/goldmark-meta v0.0.0-20191126180153-f0638e958b60
go.jolheiser.com/hcaptcha v0.0.4

2
go.sum
View File

@@ -887,6 +887,8 @@ github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1 h1:ruQGxdhGHe7FWOJPT0mKs5+pD2Xs1Bm/kdGlHO04FmM=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.3.3 h1:37BdQwPx8VOSic8eDSWee6QL9mRpZRm9VJp/QugNrW0=
github.com/yuin/goldmark v1.3.3/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691 h1:VWSxtAiQNh3zgHJpdpkpVYjTPqRE3P6UZCOPa1nRDio=
github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691/go.mod h1:YLF3kDffRfUH/bTxOxHhV6lxwIB3Vfj91rEwNMS9MXo=
github.com/yuin/goldmark-meta v0.0.0-20191126180153-f0638e958b60 h1:gZucqLjL1eDzVWrXj4uiWeMbAopJlBR2mKQAsTGdPwo=

View File

@@ -74,8 +74,79 @@ func TestAPICreatePullSuccess(t *testing.T) {
Base: "master",
Title: "create a failure pr",
})
session.MakeRequest(t, req, 201)
session.MakeRequest(t, req, http.StatusUnprocessableEntity) // second request should fail
}
func TestAPICreatePullWithFieldsSuccess(t *testing.T) {
defer prepareTestEnv(t)()
// repo10 have code, pulls units.
repo10 := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 10}).(*models.Repository)
owner10 := models.AssertExistsAndLoadBean(t, &models.User{ID: repo10.OwnerID}).(*models.User)
// repo11 only have code unit but should still create pulls
repo11 := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 11}).(*models.Repository)
owner11 := models.AssertExistsAndLoadBean(t, &models.User{ID: repo11.OwnerID}).(*models.User)
session := loginUser(t, owner11.Name)
token := getTokenForLoggedInUser(t, session)
opts := &api.CreatePullRequestOption{
Head: fmt.Sprintf("%s:master", owner11.Name),
Base: "master",
Title: "create a failure pr",
Body: "foobaaar",
Milestone: 5,
Assignees: []string{owner10.Name},
Labels: []int64{5},
}
req := NewRequestWithJSON(t, http.MethodPost, fmt.Sprintf("/api/v1/repos/%s/%s/pulls?token=%s", owner10.Name, repo10.Name, token), opts)
res := session.MakeRequest(t, req, 201)
pull := new(api.PullRequest)
DecodeJSON(t, res, pull)
assert.NotNil(t, pull.Milestone)
assert.EqualValues(t, opts.Milestone, pull.Milestone.ID)
if assert.Len(t, pull.Assignees, 1) {
assert.EqualValues(t, opts.Assignees[0], owner10.Name)
}
assert.NotNil(t, pull.Labels)
assert.EqualValues(t, opts.Labels[0], pull.Labels[0].ID)
}
func TestAPICreatePullWithFieldsFailure(t *testing.T) {
defer prepareTestEnv(t)()
// repo10 have code, pulls units.
repo10 := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 10}).(*models.Repository)
owner10 := models.AssertExistsAndLoadBean(t, &models.User{ID: repo10.OwnerID}).(*models.User)
// repo11 only have code unit but should still create pulls
repo11 := models.AssertExistsAndLoadBean(t, &models.Repository{ID: 11}).(*models.Repository)
owner11 := models.AssertExistsAndLoadBean(t, &models.User{ID: repo11.OwnerID}).(*models.User)
session := loginUser(t, owner11.Name)
token := getTokenForLoggedInUser(t, session)
opts := &api.CreatePullRequestOption{
Head: fmt.Sprintf("%s:master", owner11.Name),
Base: "master",
}
req := NewRequestWithJSON(t, http.MethodPost, fmt.Sprintf("/api/v1/repos/%s/%s/pulls?token=%s", owner10.Name, repo10.Name, token), opts)
session.MakeRequest(t, req, http.StatusUnprocessableEntity)
opts.Title = "is required"
opts.Milestone = 666
session.MakeRequest(t, req, http.StatusUnprocessableEntity)
opts.Milestone = 5
opts.Assignees = []string{"qweruqweroiuyqweoiruywqer"}
session.MakeRequest(t, req, http.StatusUnprocessableEntity)
opts.Assignees = []string{owner10.LoginName}
opts.Labels = []int64{55555}
session.MakeRequest(t, req, http.StatusUnprocessableEntity)
opts.Labels = []int64{5}
}
func TestAPIEditPull(t *testing.T) {

View File

@@ -5,10 +5,13 @@
package models
import (
"fmt"
"reflect"
"regexp"
"strings"
"testing"
"code.gitea.io/gitea/modules/setting"
"github.com/stretchr/testify/assert"
"xorm.io/builder"
)
@@ -221,6 +224,24 @@ func DeleteOrphanedLabels() error {
return nil
}
// CountOrphanedIssueLabels return count of IssueLabels witch have no label behind anymore
func CountOrphanedIssueLabels() (int64, error) {
return x.Table("issue_label").
Join("LEFT", "label", "issue_label.label_id = label.id").
Where(builder.IsNull{"label.id"}).Count()
}
// DeleteOrphanedIssueLabels delete IssueLabels witch have no label behind anymore
func DeleteOrphanedIssueLabels() error {
_, err := x.In("id", builder.Select("issue_label.id").From("issue_label").
Join("LEFT", "label", "issue_label.label_id = label.id").
Where(builder.IsNull{"label.id"})).
Delete(IssueLabel{})
return err
}
// CountOrphanedIssues count issues without a repo
func CountOrphanedIssues() (int64, error) {
return x.Table("issue").
@@ -295,3 +316,61 @@ func FixNullArchivedRepository() (int64, error) {
IsArchived: false,
})
}
// CountBadSequences looks for broken sequences from recreate-table mistakes
func CountBadSequences() (int64, error) {
if !setting.Database.UsePostgreSQL {
return 0, nil
}
sess := x.NewSession()
defer sess.Close()
var sequences []string
schema := sess.Engine().Dialect().URI().Schema
sess.Engine().SetSchema("")
if err := sess.Table("information_schema.sequences").Cols("sequence_name").Where("sequence_name LIKE 'tmp_recreate__%_id_seq%' AND sequence_catalog = ?", setting.Database.Name).Find(&sequences); err != nil {
return 0, err
}
sess.Engine().SetSchema(schema)
return int64(len(sequences)), nil
}
// FixBadSequences fixes for broken sequences from recreate-table mistakes
func FixBadSequences() error {
if !setting.Database.UsePostgreSQL {
return nil
}
sess := x.NewSession()
defer sess.Close()
if err := sess.Begin(); err != nil {
return err
}
var sequences []string
schema := sess.Engine().Dialect().URI().Schema
sess.Engine().SetSchema("")
if err := sess.Table("information_schema.sequences").Cols("sequence_name").Where("sequence_name LIKE 'tmp_recreate__%_id_seq%' AND sequence_catalog = ?", setting.Database.Name).Find(&sequences); err != nil {
return err
}
sess.Engine().SetSchema(schema)
sequenceRegexp := regexp.MustCompile(`tmp_recreate__(\w+)_id_seq.*`)
for _, sequence := range sequences {
tableName := sequenceRegexp.FindStringSubmatch(sequence)[1]
newSequenceName := tableName + "_id_seq"
if _, err := sess.Exec(fmt.Sprintf("ALTER SEQUENCE `%s` RENAME TO `%s`", sequence, newSequenceName)); err != nil {
return err
}
if _, err := sess.Exec(fmt.Sprintf("SELECT setval('%s', COALESCE((SELECT MAX(id)+1 FROM `%s`), 1), false)", newSequenceName, tableName)); err != nil {
return err
}
}
return sess.Commit()
}

View File

@@ -33,3 +33,11 @@
num_issues: 1
num_closed_issues: 0
-
id: 5
repo_id: 10
org_id: 0
name: pull-test-label
color: '#000000'
num_issues: 0
num_closed_issues: 0

View File

@@ -29,3 +29,11 @@
content: content random
is_closed: false
num_issues: 0
-
id: 5
repo_id: 10
name: milestone of repo 10
content: for testing with PRs
is_closed: false
num_issues: 0

View File

@@ -146,6 +146,7 @@
num_closed_issues: 0
num_pulls: 1
num_closed_pulls: 0
num_milestones: 1
is_mirror: false
num_forks: 1
status: 0

View File

@@ -764,3 +764,15 @@ func DeleteIssueLabel(issue *Issue, label *Label, doer *User) (err error) {
return sess.Commit()
}
func deleteLabelsByRepoID(sess Engine, repoID int64) error {
deleteCond := builder.Select("id").From("label").Where(builder.Eq{"label.repo_id": repoID})
if _, err := sess.In("label_id", deleteCond).
Delete(&IssueLabel{}); err != nil {
return err
}
_, err := sess.Delete(&Label{RepoID: repoID})
return err
}

View File

@@ -516,6 +516,31 @@ func recreateTable(sess *xorm.Session, bean interface{}) error {
return err
}
case setting.Database.UsePostgreSQL:
var originalSequences []string
type sequenceData struct {
LastValue int `xorm:"'last_value'"`
IsCalled bool `xorm:"'is_called'"`
}
sequenceMap := map[string]sequenceData{}
schema := sess.Engine().Dialect().URI().Schema
sess.Engine().SetSchema("")
if err := sess.Table("information_schema.sequences").Cols("sequence_name").Where("sequence_name LIKE ? || '_%' AND sequence_catalog = ?", tableName, setting.Database.Name).Find(&originalSequences); err != nil {
log.Error("Unable to rename %s to %s. Error: %v", tempTableName, tableName, err)
return err
}
sess.Engine().SetSchema(schema)
for _, sequence := range originalSequences {
sequenceData := sequenceData{}
if _, err := sess.Table(sequence).Cols("last_value", "is_called").Get(&sequenceData); err != nil {
log.Error("Unable to get last_value and is_called from %s. Error: %v", sequence, err)
return err
}
sequenceMap[sequence] = sequenceData
}
// CASCADE causes postgres to drop all the constraints on the old table
if _, err := sess.Exec(fmt.Sprintf("DROP TABLE `%s` CASCADE", tableName)); err != nil {
log.Error("Unable to drop old table %s. Error: %v", tableName, err)
@@ -529,7 +554,6 @@ func recreateTable(sess *xorm.Session, bean interface{}) error {
}
var indices []string
schema := sess.Engine().Dialect().URI().Schema
sess.Engine().SetSchema("")
if err := sess.Table("pg_indexes").Cols("indexname").Where("tablename = ? ", tableName).Find(&indices); err != nil {
log.Error("Unable to rename %s to %s. Error: %v", tempTableName, tableName, err)
@@ -545,6 +569,43 @@ func recreateTable(sess *xorm.Session, bean interface{}) error {
}
}
var sequences []string
sess.Engine().SetSchema("")
if err := sess.Table("information_schema.sequences").Cols("sequence_name").Where("sequence_name LIKE 'tmp_recreate__' || ? || '_%' AND sequence_catalog = ?", tableName, setting.Database.Name).Find(&sequences); err != nil {
log.Error("Unable to rename %s to %s. Error: %v", tempTableName, tableName, err)
return err
}
sess.Engine().SetSchema(schema)
for _, sequence := range sequences {
newSequenceName := strings.Replace(sequence, "tmp_recreate__", "", 1)
if _, err := sess.Exec(fmt.Sprintf("ALTER SEQUENCE `%s` RENAME TO `%s`", sequence, newSequenceName)); err != nil {
log.Error("Unable to rename %s sequence to %s. Error: %v", sequence, newSequenceName, err)
return err
}
val, ok := sequenceMap[newSequenceName]
if newSequenceName == tableName+"_id_seq" {
if ok && val.LastValue != 0 {
if _, err := sess.Exec(fmt.Sprintf("SELECT setval('%s', %d, %t)", newSequenceName, val.LastValue, val.IsCalled)); err != nil {
log.Error("Unable to reset %s to %d. Error: %v", newSequenceName, val, err)
return err
}
} else {
// We're going to try to guess this
if _, err := sess.Exec(fmt.Sprintf("SELECT setval('%s', COALESCE((SELECT MAX(id)+1 FROM `%s`), 1), false)", newSequenceName, tableName)); err != nil {
log.Error("Unable to reset %s. Error: %v", newSequenceName, err)
return err
}
}
} else if ok {
if _, err := sess.Exec(fmt.Sprintf("SELECT setval('%s', %d, %t)", newSequenceName, val.LastValue, val.IsCalled)); err != nil {
log.Error("Unable to reset %s to %d. Error: %v", newSequenceName, val, err)
return err
}
}
}
case setting.Database.UseMSSQL:
// MSSQL will drop all the constraints on the old table
if _, err := sess.Exec(fmt.Sprintf("DROP TABLE `%s`", tableName)); err != nil {

View File

@@ -1729,6 +1729,10 @@ func DeleteRepository(doer *User, uid, repoID int64) error {
return fmt.Errorf("deleteBeans: %v", err)
}
if err := deleteLabelsByRepoID(sess, repoID); err != nil {
return err
}
// Delete Issues and related objects
var attachmentPaths []string
if attachmentPaths, err = deleteIssuesByRepoID(sess, repoID); err != nil {

View File

@@ -727,6 +727,7 @@ var (
"assets",
"attachments",
"avatars",
"captcha",
"commits",
"debug",
"error",

View File

@@ -13,6 +13,10 @@ import (
// ToUser convert models.User to api.User
// signed shall only be set if requester is logged in. authed shall only be set if user is site admin or user himself
func ToUser(user *models.User, signed, authed bool) *api.User {
if user == nil {
return nil
}
result := &api.User{
ID: user.ID,
UserName: user.Name,

View File

@@ -30,6 +30,9 @@ var (
// aliasMap provides a map of the alias to its emoji data.
aliasMap map[string]int
// emptyReplacer is the string replacer for emoji codes.
emptyReplacer *strings.Replacer
// codeReplacer is the string replacer for emoji codes.
codeReplacer *strings.Replacer
@@ -49,6 +52,7 @@ func loadMap() {
// process emoji codes and aliases
codePairs := make([]string, 0)
emptyPairs := make([]string, 0)
aliasPairs := make([]string, 0)
// sort from largest to small so we match combined emoji first
@@ -64,6 +68,7 @@ func loadMap() {
// setup codes
codeMap[e.Emoji] = i
codePairs = append(codePairs, e.Emoji, ":"+e.Aliases[0]+":")
emptyPairs = append(emptyPairs, e.Emoji, e.Emoji)
// setup aliases
for _, a := range e.Aliases {
@@ -77,6 +82,7 @@ func loadMap() {
}
// create replacers
emptyReplacer = strings.NewReplacer(emptyPairs...)
codeReplacer = strings.NewReplacer(codePairs...)
aliasReplacer = strings.NewReplacer(aliasPairs...)
})
@@ -127,38 +133,53 @@ func ReplaceAliases(s string) string {
return aliasReplacer.Replace(s)
}
type rememberSecondWriteWriter struct {
pos int
idx int
end int
writecount int
}
func (n *rememberSecondWriteWriter) Write(p []byte) (int, error) {
n.writecount++
if n.writecount == 2 {
n.idx = n.pos
n.end = n.pos + len(p)
}
n.pos += len(p)
return len(p), nil
}
func (n *rememberSecondWriteWriter) WriteString(s string) (int, error) {
n.writecount++
if n.writecount == 2 {
n.idx = n.pos
n.end = n.pos + len(s)
}
n.pos += len(s)
return len(s), nil
}
// FindEmojiSubmatchIndex returns index pair of longest emoji in a string
func FindEmojiSubmatchIndex(s string) []int {
loadMap()
found := make(map[int]int)
keys := make([]int, 0)
secondWriteWriter := rememberSecondWriteWriter{}
//see if there are any emoji in string before looking for position of specific ones
//no performance difference when there is a match but 10x faster when there are not
if s == ReplaceCodes(s) {
// A faster and clean implementation would copy the trie tree formation in strings.NewReplacer but
// we can be lazy here.
//
// The implementation of strings.Replacer.WriteString is such that the first index of the emoji
// submatch is simply the second thing that is written to WriteString in the writer.
//
// Therefore we can simply take the index of the second write as our first emoji
//
// FIXME: just copy the trie implementation from strings.NewReplacer
_, _ = emptyReplacer.WriteString(&secondWriteWriter, s)
// if we wrote less than twice then we never "replaced"
if secondWriteWriter.writecount < 2 {
return nil
}
// get index of first emoji occurrence while also checking for longest combination
for j := range GemojiData {
i := strings.Index(s, GemojiData[j].Emoji)
if i != -1 {
if _, ok := found[i]; !ok {
if len(keys) == 0 || i < keys[0] {
found[i] = j
keys = []int{i}
}
if i == 0 {
break
}
}
}
}
if len(keys) > 0 {
index := keys[0]
return []int{index, index + len(GemojiData[found[index]].Emoji)}
}
return nil
return []int{secondWriteWriter.idx, secondWriteWriter.end}
}

View File

@@ -8,6 +8,8 @@ package emoji
import (
"reflect"
"testing"
"github.com/stretchr/testify/assert"
)
func TestDumpInfo(t *testing.T) {
@@ -65,3 +67,34 @@ func TestReplacers(t *testing.T) {
}
}
}
func TestFindEmojiSubmatchIndex(t *testing.T) {
type testcase struct {
teststring string
expected []int
}
testcases := []testcase{
{
"\U0001f44d",
[]int{0, len("\U0001f44d")},
},
{
"\U0001f44d +1 \U0001f44d \U0001f37a",
[]int{0, 4},
},
{
" \U0001f44d",
[]int{1, 1 + len("\U0001f44d")},
},
{
string([]byte{'\u0001'}) + "\U0001f44d",
[]int{1, 1 + len("\U0001f44d")},
},
}
for _, kase := range testcases {
actual := FindEmojiSubmatchIndex(kase.teststring)
assert.Equal(t, kase.expected, actual)
}
}

View File

@@ -298,19 +298,27 @@ func RenderEmoji(
return ctx.postProcess(rawHTML)
}
var tagCleaner = regexp.MustCompile(`<((?:/?\w+/\w+)|(?:/[\w ]+/)|(/?[hH][tT][mM][lL][ />])|(/?[hH][eE][aA][dD][ />]))`)
var nulCleaner = strings.NewReplacer("\000", "")
func (ctx *postProcessCtx) postProcess(rawHTML []byte) ([]byte, error) {
if ctx.procs == nil {
ctx.procs = defaultProcessors
}
// give a generous extra 50 bytes
res := make([]byte, 0, len(rawHTML)+50)
res = append(res, "<html><body>"...)
res = append(res, rawHTML...)
res = append(res, "</body></html>"...)
res := bytes.NewBuffer(make([]byte, 0, len(rawHTML)+50))
// prepend "<html><body>"
_, _ = res.WriteString("<html><body>")
// Strip out nuls - they're always invalid
_, _ = res.Write(tagCleaner.ReplaceAll([]byte(nulCleaner.Replace(string(rawHTML))), []byte("&lt;$1")))
// close the tags
_, _ = res.WriteString("</body></html>")
// parse the HTML
nodes, err := html.ParseFragment(bytes.NewReader(res), nil)
nodes, err := html.ParseFragment(res, nil)
if err != nil {
return nil, &postProcessError{"invalid HTML", err}
}
@@ -347,17 +355,17 @@ func (ctx *postProcessCtx) postProcess(rawHTML []byte) ([]byte, error) {
// Create buffer in which the data will be placed again. We know that the
// length will be at least that of res; to spare a few alloc+copy, we
// reuse res, resetting its length to 0.
buf := bytes.NewBuffer(res[:0])
res.Reset()
// Render everything to buf.
for _, node := range nodes {
err = html.Render(buf, node)
err = html.Render(res, node)
if err != nil {
return nil, &postProcessError{"error rendering processed HTML", err}
}
}
// Everything done successfully, return parsed data.
return buf.Bytes(), nil
return res.Bytes(), nil
}
func (ctx *postProcessCtx) visitNode(node *html.Node, visitText bool) {

View File

@@ -10,6 +10,7 @@ import (
"regexp"
"strings"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/markup"
"code.gitea.io/gitea/modules/markup/common"
"code.gitea.io/gitea/modules/setting"
@@ -76,6 +77,12 @@ func (g *ASTTransformer) Transform(node *ast.Document, reader text.Reader, pc pa
header.ID = util.BytesToReadOnlyString(id.([]byte))
}
toc = append(toc, header)
} else {
for _, attr := range v.Attributes() {
if _, ok := attr.Value.([]byte); !ok {
v.SetAttribute(attr.Name, []byte(fmt.Sprintf("%v", attr.Value)))
}
}
}
case *ast.Image:
// Images need two things:
@@ -101,11 +108,41 @@ func (g *ASTTransformer) Transform(node *ast.Document, reader text.Reader, pc pa
parent := n.Parent()
// Create a link around image only if parent is not already a link
if _, ok := parent.(*ast.Link); !ok && parent != nil {
next := n.NextSibling()
// Create a link wrapper
wrap := ast.NewLink()
wrap.Destination = link
wrap.Title = v.Title
// Duplicate the current image node
image := ast.NewImage(ast.NewLink())
image.Destination = link
image.Title = v.Title
for _, attr := range v.Attributes() {
image.SetAttribute(attr.Name, attr.Value)
}
for child := v.FirstChild(); child != nil; {
next := child.NextSibling()
image.AppendChild(image, child)
child = next
}
// Append our duplicate image to the wrapper link
wrap.AppendChild(wrap, image)
// Wire in the next sibling
wrap.SetNextSibling(next)
// Replace the current node with the wrapper link
parent.ReplaceChild(parent, n, wrap)
wrap.AppendChild(wrap, n)
// But most importantly ensure the next sibling is still on the old image too
v.SetNextSibling(next)
} else {
log.Debug("ast.Image: %s has parent: %v", link, parent)
}
case *ast.Link:
// Links need their href to munged to be a real value

View File

@@ -6,7 +6,8 @@
package markdown
import (
"bytes"
"fmt"
"io"
"strings"
"sync"
@@ -18,7 +19,7 @@ import (
chromahtml "github.com/alecthomas/chroma/formatters/html"
"github.com/yuin/goldmark"
"github.com/yuin/goldmark-highlighting"
highlighting "github.com/yuin/goldmark-highlighting"
meta "github.com/yuin/goldmark-meta"
"github.com/yuin/goldmark/extension"
"github.com/yuin/goldmark/parser"
@@ -34,6 +35,44 @@ var urlPrefixKey = parser.NewContextKey()
var isWikiKey = parser.NewContextKey()
var renderMetasKey = parser.NewContextKey()
type closesWithError interface {
io.WriteCloser
CloseWithError(err error) error
}
type limitWriter struct {
w closesWithError
sum int64
limit int64
}
// Write implements the standard Write interface:
func (l *limitWriter) Write(data []byte) (int, error) {
leftToWrite := l.limit - l.sum
if leftToWrite < int64(len(data)) {
n, err := l.w.Write(data[:leftToWrite])
l.sum += int64(n)
if err != nil {
return n, err
}
_ = l.w.Close()
return n, fmt.Errorf("Rendered content too large - truncating render")
}
n, err := l.w.Write(data)
l.sum += int64(n)
return n, err
}
// Close closes the writer
func (l *limitWriter) Close() error {
return l.w.Close()
}
// CloseWithError closes the writer
func (l *limitWriter) CloseWithError(err error) error {
return l.w.CloseWithError(err)
}
// NewGiteaParseContext creates a parser.Context with the gitea context set
func NewGiteaParseContext(urlPrefix string, metas map[string]string, isWiki bool) parser.Context {
pc := parser.NewContext(parser.WithIDs(newPrefixedIDs()))
@@ -43,8 +82,8 @@ func NewGiteaParseContext(urlPrefix string, metas map[string]string, isWiki bool
return pc
}
// render renders Markdown to HTML without handling special links.
func render(body []byte, urlPrefix string, metas map[string]string, wikiMarkdown bool) []byte {
// actualRender renders Markdown to HTML without handling special links.
func actualRender(body []byte, urlPrefix string, metas map[string]string, wikiMarkdown bool) []byte {
once.Do(func() {
converter = goldmark.New(
goldmark.WithExtensions(extension.Table,
@@ -119,12 +158,57 @@ func render(body []byte, urlPrefix string, metas map[string]string, wikiMarkdown
})
pc := NewGiteaParseContext(urlPrefix, metas, wikiMarkdown)
var buf bytes.Buffer
if err := converter.Convert(giteautil.NormalizeEOL(body), &buf, parser.WithContext(pc)); err != nil {
log.Error("Unable to render: %v", err)
rd, wr := io.Pipe()
defer func() {
_ = rd.Close()
_ = wr.Close()
}()
lw := &limitWriter{
w: wr,
limit: setting.UI.MaxDisplayFileSize * 3,
}
return markup.SanitizeReader(&buf).Bytes()
// FIXME: should we include a timeout that closes the pipe to abort the parser and sanitizer if it takes too long?
go func() {
defer func() {
err := recover()
if err == nil {
return
}
log.Warn("Unable to render markdown due to panic in goldmark: %v", err)
if log.IsDebug() {
log.Debug("Panic in markdown: %v\n%s", err, string(log.Stack(2)))
}
_ = lw.CloseWithError(fmt.Errorf("%v", err))
}()
pc := NewGiteaParseContext(urlPrefix, metas, wikiMarkdown)
if err := converter.Convert(giteautil.NormalizeEOL(body), lw, parser.WithContext(pc)); err != nil {
log.Error("Unable to render: %v", err)
_ = lw.CloseWithError(err)
return
}
_ = lw.Close()
}()
return markup.SanitizeReader(rd).Bytes()
}
func render(body []byte, urlPrefix string, metas map[string]string, wikiMarkdown bool) (ret []byte) {
defer func() {
err := recover()
if err == nil {
return
}
log.Warn("Unable to render markdown due to panic in goldmark - will return sanitized raw bytes")
if log.IsDebug() {
log.Debug("Panic in markdown: %v\n%s", err, string(log.Stack(2)))
}
ret = markup.SanitizeBytes(body)
}()
return actualRender(body, urlPrefix, metas, wikiMarkdown)
}
var (

View File

@@ -308,3 +308,34 @@ func TestRender_RenderParagraphs(t *testing.T) {
test(t, "A\n\nB\nC\n", 2)
test(t, "A\n\n\nB\nC\n", 2)
}
func TestMarkdownRenderRaw(t *testing.T) {
testcases := [][]byte{
{ // clusterfuzz_testcase_minimized_fuzz_markdown_render_raw_6267570554535936
0x2a, 0x20, 0x2d, 0x0a, 0x09, 0x20, 0x60, 0x5b, 0x0a, 0x09, 0x20, 0x60,
0x5b,
},
{ // clusterfuzz_testcase_minimized_fuzz_markdown_render_raw_6278827345051648
0x2d, 0x20, 0x2d, 0x0d, 0x09, 0x60, 0x0d, 0x09, 0x60,
},
{ // clusterfuzz_testcase_minimized_fuzz_markdown_render_raw_6016973788020736[] = {
0x7b, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x3d, 0x35, 0x7d, 0x0a, 0x3d,
},
}
for _, testcase := range testcases {
_ = RenderRaw(testcase, "", false)
}
}
func TestRenderSiblingImages_Issue12925(t *testing.T) {
testcase := `![image1](/image1)
![image2](/image2)
`
expected := `<p><a href="/image1" rel="nofollow"><img src="/image1" alt="image1"></a><br>
<a href="/image2" rel="nofollow"><img src="/image2" alt="image2"></a></p>
`
res := string(RenderRaw([]byte(testcase), "", false))
assert.Equal(t, expected, res)
}

View File

@@ -52,6 +52,13 @@ func isMigrateURLAllowed(remoteURL string) error {
}
}
if u.Host == "" {
if !setting.ImportLocalPaths {
return &models.ErrMigrationNotAllowed{Host: "<LOCAL_FILESYSTEM>"}
}
return nil
}
if !setting.Migrations.AllowLocalNetworks {
addrList, err := net.LookupIP(strings.Split(u.Host, ":")[0])
if err != nil {

View File

@@ -31,4 +31,16 @@ func TestMigrateWhiteBlocklist(t *testing.T) {
err = isMigrateURLAllowed("https://github.com/go-gitea/gitea.git")
assert.Error(t, err)
old := setting.ImportLocalPaths
setting.ImportLocalPaths = false
err = isMigrateURLAllowed("/home/foo/bar/goo")
assert.Error(t, err)
setting.ImportLocalPaths = true
err = isMigrateURLAllowed("/home/foo/bar/goo")
assert.NoError(t, err)
setting.ImportLocalPaths = old
}

View File

@@ -120,7 +120,6 @@ func UploadRepoFiles(repo *models.Repository, doer *models.User, opts *UploadRep
return err
}
infos[i] = uploadInfo
} else if objectHash, err = t.HashObject(file); err != nil {
return err
}
@@ -128,7 +127,6 @@ func UploadRepoFiles(repo *models.Repository, doer *models.User, opts *UploadRep
// Add the object to the index
if err := t.AddObjectToIndex("100644", objectHash, path.Join(opts.TreePath, uploadInfo.upload.Name)); err != nil {
return err
}
}
@@ -165,28 +163,10 @@ func UploadRepoFiles(repo *models.Repository, doer *models.User, opts *UploadRep
// OK now we can insert the data into the store - there's no way to clean up the store
// once it's in there, it's in there.
contentStore := &lfs.ContentStore{ObjectStorage: storage.LFS}
for _, uploadInfo := range infos {
if uploadInfo.lfsMetaObject == nil {
continue
}
exist, err := contentStore.Exists(uploadInfo.lfsMetaObject)
if err != nil {
for _, info := range infos {
if err := uploadToLFSContentStore(info, contentStore); err != nil {
return cleanUpAfterFailure(&infos, t, err)
}
if !exist {
file, err := os.Open(uploadInfo.upload.LocalPath())
if err != nil {
return cleanUpAfterFailure(&infos, t, err)
}
defer file.Close()
// FIXME: Put regenerates the hash and copies the file over.
// I guess this strictly ensures the soundness of the store but this is inefficient.
if err := contentStore.Put(uploadInfo.lfsMetaObject, file); err != nil {
// OK Now we need to cleanup
// Can't clean up the store, once uploaded there they're there.
return cleanUpAfterFailure(&infos, t, err)
}
}
}
// Then push this tree to NewBranch
@@ -196,3 +176,29 @@ func UploadRepoFiles(repo *models.Repository, doer *models.User, opts *UploadRep
return models.DeleteUploads(uploads...)
}
func uploadToLFSContentStore(info uploadInfo, contentStore *lfs.ContentStore) error {
if info.lfsMetaObject == nil {
return nil
}
exist, err := contentStore.Exists(info.lfsMetaObject)
if err != nil {
return err
}
if !exist {
file, err := os.Open(info.upload.LocalPath())
if err != nil {
return err
}
defer file.Close()
// FIXME: Put regenerates the hash and copies the file over.
// I guess this strictly ensures the soundness of the store but this is inefficient.
if err := contentStore.Put(info.lfsMetaObject, file); err != nil {
// OK Now we need to cleanup
// Can't clean up the store, once uploaded there they're there.
return err
}
}
return nil
}

View File

@@ -293,7 +293,6 @@ func CreatePullRequest(ctx *context.APIContext, form api.CreatePullRequestOption
var (
repo = ctx.Repo.Repository
labelIDs []int64
assigneeID int64
milestoneID int64
)
@@ -354,7 +353,7 @@ func CreatePullRequest(ctx *context.APIContext, form api.CreatePullRequestOption
}
if form.Milestone > 0 {
milestone, err := models.GetMilestoneByRepoID(ctx.Repo.Repository.ID, milestoneID)
milestone, err := models.GetMilestoneByRepoID(ctx.Repo.Repository.ID, form.Milestone)
if err != nil {
if models.IsErrMilestoneNotExist(err) {
ctx.NotFound()
@@ -378,7 +377,6 @@ func CreatePullRequest(ctx *context.APIContext, form api.CreatePullRequestOption
PosterID: ctx.User.ID,
Poster: ctx.User,
MilestoneID: milestoneID,
AssigneeID: assigneeID,
IsPull: true,
Content: form.Body,
DeadlineUnix: deadlineUnix,

View File

@@ -539,6 +539,10 @@ func updateBasicProperties(ctx *context.APIContext, opts api.EditRepoOption) err
if opts.Private != nil {
// Visibility of forked repository is forced sync with base repository.
if repo.IsFork {
if err := repo.GetBaseRepo(); err != nil {
ctx.Error(http.StatusInternalServerError, "Unable to load base repository", err)
return err
}
*opts.Private = repo.BaseRepo.IsPrivate
}

View File

@@ -13,6 +13,7 @@ import (
"net/http"
"os"
"path"
"path/filepath"
"strings"
"text/template"
"time"
@@ -152,12 +153,21 @@ func storageHandler(storageSetting setting.Storage, prefix string, objStore stor
return
}
if !strings.HasPrefix(req.URL.RequestURI(), "/"+prefix) {
prefix := strings.Trim(prefix, "/")
if !strings.HasPrefix(req.URL.EscapedPath(), "/"+prefix+"/") {
return
}
rPath := strings.TrimPrefix(req.URL.EscapedPath(), "/"+prefix+"/")
rPath := strings.TrimPrefix(req.URL.RequestURI(), "/"+prefix)
rPath = strings.TrimPrefix(rPath, "/")
if rPath == "" {
ctx.Error(404, "file not found")
return
}
rPath = path.Clean("/" + filepath.ToSlash(rPath))
rPath = rPath[1:]
//If we have matched and access to release or issue
fr, err := objStore.Open(rPath)
if err != nil {

View File

@@ -1282,6 +1282,16 @@ func CommentAsDiff(c *models.Comment) (*Diff, error) {
// CommentMustAsDiff executes AsDiff and logs the error instead of returning
func CommentMustAsDiff(c *models.Comment) *Diff {
if c == nil {
return nil
}
defer func() {
if err := recover(); err != nil {
stack := log.Stack(2)
log.Error("PANIC whilst retrieving diff for comment[%d] Error: %v\nStack: %s", c.ID, err, stack)
panic(fmt.Errorf("PANIC whilst retrieving diff for comment[%d] Error: %v\nStack: %s", c.ID, err, stack))
}
}()
diff, err := CommentAsDiff(c)
if err != nil {
log.Warn("CommentMustAsDiff: %v", err)

View File

@@ -12,8 +12,10 @@
</head>
<body>
<p><b>@{{.Release.Publisher.Name}}</b> released <a href="{{.Release.HTMLURL}}">{{.Release.TagName}}</a>
in <a href="{{AppUrl}}{{.Release.Repo.OwnerName}}/{{.Release.Repo.Name}}">{{.Release.Repo.FullName}} </p>
<p>
<b>@{{.Release.Publisher.Name}}</b> released <a href="{{.Release.HTMLURL}}">{{.Release.TagName}}</a>
in <a href="{{AppUrl}}{{.Release.Repo.OwnerName}}/{{.Release.Repo.Name}}">{{.Release.Repo.FullName}}</a>
</p>
<h4>Title: {{.Release.Title}}</h4>
<p>
Note: <br>

View File

@@ -4,8 +4,8 @@
<div class="ui container">
<div class="ui three column stackable grid">
<div class="column">
<h3>{{.Milestone.Name}}</h3>
<div class="content">
<h1>{{.Milestone.Name}}</h1>
<div class="markdown content">
{{.Milestone.RenderedContent|Str2html}}
</div>
</div>
@@ -63,7 +63,7 @@
<span class="info">{{.i18n.Tr "repo.issues.filter_label_exclude" | Safe}}</span>
<a class="item" href="{{$.Link}}?q={{$.Keyword}}&type={{$.ViewType}}&sort={{$.SortType}}&state={{$.State}}&assignee={{$.AssigneeID}}">{{.i18n.Tr "repo.issues.filter_label_no_select"}}</a>
{{range .Labels}}
<a class="item label-filter-item" href="{{$.Link}}?q={{$.Keyword}}&type={{$.ViewType}}&sort={{$.SortType}}&state={{$.State}}&labels={{.ID}}&assignee={{$.AssigneeID}}" data-label-id="{{.ID}}">{{if .IsExcluded}}{{svg "octicon-circle-slash"}}{{else if contain $.SelLabelIDs .ID}}{{svg "octicon-check"}}{{end}}<span class="label color" style="background-color: {{.Color}}"></span> {{.Name | RenderEmoji}}</a>
<a class="item label-filter-item" href="{{$.Link}}?q={{$.Keyword}}&type={{$.ViewType}}&sort={{$.SortType}}&state={{$.State}}&labels={{.QueryString}}&assignee={{$.AssigneeID}}" data-label-id="{{.ID}}">{{if .IsExcluded}}{{svg "octicon-circle-slash"}}{{else if contain $.SelLabelIDs .ID}}{{svg "octicon-check"}}{{end}}<span class="label color" style="background-color: {{.Color}}"></span> {{.Name | RenderEmoji}}</a>
{{end}}
</div>
</div>

View File

@@ -43,7 +43,7 @@
<div class="milestone list">
{{range .Milestones}}
<li class="item">
{{svg "octicon-milestone"}} <a href="{{$.RepoLink}}/milestone/{{.ID}}">{{.Name}}</a>
{{svg "octicon-milestone" 16 "mr-2"}} <a href="{{$.RepoLink}}/milestone/{{.ID}}">{{.Name}}</a>
<div class="ui right green progress" data-percent="{{.Completeness}}">
<div class="bar" {{if not .Completeness}}style="background-color: transparent"{{end}}>
<div class="progress"></div>
@@ -80,7 +80,7 @@
</div>
{{end}}
{{if .Content}}
<div class="content">
<div class="markdown content">
{{.RenderedContent|Str2html}}
</div>
{{end}}

View File

@@ -29,7 +29,7 @@
</div>
<div class="header-right actions df ac">
{{if not $.Repository.IsArchived}}
{{if or (and (eq .PosterID .Issue.PosterID) (eq .Issue.OriginalAuthorID 0)) (eq .Issue.OriginalAuthorID .OriginalAuthorID) }}
{{if or (and (eq .PosterID .Issue.PosterID) (eq .Issue.OriginalAuthorID 0)) (and (eq .Issue.OriginalAuthorID .OriginalAuthorID) (not (eq .OriginalAuthorID 0))) }}
<div class="item tag">
{{$.i18n.Tr "repo.issues.poster"}}
</div>

View File

@@ -1,7 +1,7 @@
goldmark
==========================================
[![http://godoc.org/github.com/yuin/goldmark](https://godoc.org/github.com/yuin/goldmark?status.svg)](http://godoc.org/github.com/yuin/goldmark)
[![https://pkg.go.dev/github.com/yuin/goldmark](https://pkg.go.dev/badge/github.com/yuin/goldmark.svg)](https://pkg.go.dev/github.com/yuin/goldmark)
[![https://github.com/yuin/goldmark/actions?query=workflow:test](https://github.com/yuin/goldmark/workflows/test/badge.svg?branch=master&event=push)](https://github.com/yuin/goldmark/actions?query=workflow:test)
[![https://coveralls.io/github/yuin/goldmark](https://coveralls.io/repos/github/yuin/goldmark/badge.svg?branch=master)](https://coveralls.io/github/yuin/goldmark)
[![https://goreportcard.com/report/github.com/yuin/goldmark](https://goreportcard.com/badge/github.com/yuin/goldmark)](https://goreportcard.com/report/github.com/yuin/goldmark)
@@ -173,6 +173,7 @@ Parser and Renderer options
- This extension enables Table, Strikethrough, Linkify and TaskList.
- This extension does not filter tags defined in [6.11: Disallowed Raw HTML (extension)](https://github.github.com/gfm/#disallowed-raw-html-extension-).
If you need to filter HTML tags, see [Security](#security).
- If you need to parse github emojis, you can use [goldmark-emoji](https://github.com/yuin/goldmark-emoji) extension.
- `extension.DefinitionList`
- [PHP Markdown Extra: Definition lists](https://michelf.ca/projects/php-markdown/extra/#def-list)
- `extension.Footnote`
@@ -279,13 +280,96 @@ markdown := goldmark.New(
[]byte("https:"),
}),
extension.WithLinkifyURLRegexp(
xurls.Strict(),
xurls.Strict,
),
),
),
)
```
### Footnotes extension
The Footnote extension implements [PHP Markdown Extra: Footnotes](https://michelf.ca/projects/php-markdown/extra/#footnotes).
This extension has some options:
| Functional option | Type | Description |
| ----------------- | ---- | ----------- |
| `extension.WithFootnoteIDPrefix` | `[]byte` | a prefix for the id attributes.|
| `extension.WithFootnoteIDPrefixFunction` | `func(gast.Node) []byte` | a function that determines the id attribute for given Node.|
| `extension.WithFootnoteLinkTitle` | `[]byte` | an optional title attribute for footnote links.|
| `extension.WithFootnoteBacklinkTitle` | `[]byte` | an optional title attribute for footnote backlinks. |
| `extension.WithFootnoteLinkClass` | `[]byte` | a class for footnote links. This defaults to `footnote-ref`. |
| `extension.WithFootnoteBacklinkClass` | `[]byte` | a class for footnote backlinks. This defaults to `footnote-backref`. |
| `extension.WithFootnoteBacklinkHTML` | `[]byte` | a class for footnote backlinks. This defaults to `&#x21a9;&#xfe0e;`. |
Some options can have special substitutions. Occurances of “^^” in the string will be replaced by the corresponding footnote number in the HTML output. Occurances of “%%” will be replaced by a number for the reference (footnotes can have multiple references).
`extension.WithFootnoteIDPrefix` and `extension.WithFootnoteIDPrefixFunction` are useful if you have multiple Markdown documents displayed inside one HTML document to avoid footnote ids to clash each other.
`extension.WithFootnoteIDPrefix` sets fixed id prefix, so you may write codes like the following:
```go
for _, path := range files {
source := readAll(path)
prefix := getPrefix(path)
markdown := goldmark.New(
goldmark.WithExtensions(
NewFootnote(
WithFootnoteIDPrefix([]byte(path)),
),
),
)
var b bytes.Buffer
err := markdown.Convert(source, &b)
if err != nil {
t.Error(err.Error())
}
}
```
`extension.WithFootnoteIDPrefixFunction` determines an id prefix by calling given function, so you may write codes like the following:
```go
markdown := goldmark.New(
goldmark.WithExtensions(
NewFootnote(
WithFootnoteIDPrefixFunction(func(n gast.Node) []byte {
v, ok := n.OwnerDocument().Meta()["footnote-prefix"]
if ok {
return util.StringToReadOnlyBytes(v.(string))
}
return nil
}),
),
),
)
for _, path := range files {
source := readAll(path)
var b bytes.Buffer
doc := markdown.Parser().Parse(text.NewReader(source))
doc.Meta()["footnote-prefix"] = getPrefix(path)
err := markdown.Renderer().Render(&b, source, doc)
}
```
You can use [goldmark-meta](https://github.com/yuin/goldmark-meta) to define a id prefix in the markdown document:
```markdown
---
title: document title
slug: article1
footnote-prefix: article1
---
# My article
```
Security
--------------------
By default, goldmark does not render raw HTML or potentially-dangerous URLs.
@@ -336,6 +420,8 @@ Extensions
extension for the goldmark Markdown parser.
- [goldmark-highlighting](https://github.com/yuin/goldmark-highlighting): A syntax-highlighting extension
for the goldmark markdown parser.
- [goldmark-emoji](https://github.com/yuin/goldmark-emoji): An emoji
extension for the goldmark Markdown parser.
- [goldmark-mathjax](https://github.com/litao91/goldmark-mathjax): Mathjax support for the goldmark markdown parser
goldmark internal(for extension developers)

View File

@@ -45,11 +45,6 @@ type Attribute struct {
Value interface{}
}
var attrNameIDS = []byte("#")
var attrNameID = []byte("id")
var attrNameClassS = []byte(".")
var attrNameClass = []byte("class")
// A Node interface defines basic AST node functionalities.
type Node interface {
// Type returns a type of this node.
@@ -116,6 +111,11 @@ type Node interface {
// tail of the children.
InsertAfter(self, v1, insertee Node)
// OwnerDocument returns this node's owner document.
// If this node is not a child of the Document node, OwnerDocument
// returns nil.
OwnerDocument() *Document
// Dump dumps an AST tree structure to stdout.
// This function completely aimed for debugging.
// level is a indent level. Implementer should indent informations with
@@ -169,7 +169,7 @@ type Node interface {
RemoveAttributes()
}
// A BaseNode struct implements the Node interface.
// A BaseNode struct implements the Node interface partialliy.
type BaseNode struct {
firstChild Node
lastChild Node
@@ -358,6 +358,22 @@ func (n *BaseNode) InsertBefore(self, v1, insertee Node) {
}
}
// OwnerDocument implements Node.OwnerDocument
func (n *BaseNode) OwnerDocument() *Document {
d := n.Parent()
for {
p := d.Parent()
if p == nil {
if v, ok := d.(*Document); ok {
return v
}
break
}
d = p
}
return nil
}
// Text implements Node.Text .
func (n *BaseNode) Text(source []byte) []byte {
var buf bytes.Buffer

View File

@@ -7,7 +7,7 @@ import (
textm "github.com/yuin/goldmark/text"
)
// A BaseBlock struct implements the Node interface.
// A BaseBlock struct implements the Node interface partialliy.
type BaseBlock struct {
BaseNode
blankPreviousLines bool
@@ -50,6 +50,8 @@ func (b *BaseBlock) SetLines(v *textm.Segments) {
// A Document struct is a root node of Markdown text.
type Document struct {
BaseBlock
meta map[string]interface{}
}
// KindDocument is a NodeKind of the Document node.
@@ -70,10 +72,29 @@ func (n *Document) Kind() NodeKind {
return KindDocument
}
// OwnerDocument implements Node.OwnerDocument
func (n *Document) OwnerDocument() *Document {
return n
}
// Meta returns metadata of this document.
func (n *Document) Meta() map[string]interface{} {
if n.meta == nil {
n.meta = map[string]interface{}{}
}
return n.meta
}
// SetMeta sets given metadata to this document.
func (n *Document) SetMeta(meta map[string]interface{}) {
n.meta = meta
}
// NewDocument returns a new Document node.
func NewDocument() *Document {
return &Document{
BaseBlock: BaseBlock{},
meta: nil,
}
}

View File

@@ -8,7 +8,7 @@ import (
"github.com/yuin/goldmark/util"
)
// A BaseInline struct implements the Node interface.
// A BaseInline struct implements the Node interface partialliy.
type BaseInline struct {
BaseNode
}

View File

@@ -2,6 +2,7 @@ package ast
import (
"fmt"
gast "github.com/yuin/goldmark/ast"
)
@@ -9,13 +10,15 @@ import (
// (PHP Markdown Extra) text.
type FootnoteLink struct {
gast.BaseInline
Index int
Index int
RefCount int
}
// Dump implements Node.Dump.
func (n *FootnoteLink) Dump(source []byte, level int) {
m := map[string]string{}
m["Index"] = fmt.Sprintf("%v", n.Index)
m["RefCount"] = fmt.Sprintf("%v", n.RefCount)
gast.DumpHelper(n, source, level, m, nil)
}
@@ -30,36 +33,40 @@ func (n *FootnoteLink) Kind() gast.NodeKind {
// NewFootnoteLink returns a new FootnoteLink node.
func NewFootnoteLink(index int) *FootnoteLink {
return &FootnoteLink{
Index: index,
Index: index,
RefCount: 0,
}
}
// A FootnoteBackLink struct represents a link to a footnote of Markdown
// A FootnoteBacklink struct represents a link to a footnote of Markdown
// (PHP Markdown Extra) text.
type FootnoteBackLink struct {
type FootnoteBacklink struct {
gast.BaseInline
Index int
Index int
RefCount int
}
// Dump implements Node.Dump.
func (n *FootnoteBackLink) Dump(source []byte, level int) {
func (n *FootnoteBacklink) Dump(source []byte, level int) {
m := map[string]string{}
m["Index"] = fmt.Sprintf("%v", n.Index)
m["RefCount"] = fmt.Sprintf("%v", n.RefCount)
gast.DumpHelper(n, source, level, m, nil)
}
// KindFootnoteBackLink is a NodeKind of the FootnoteBackLink node.
var KindFootnoteBackLink = gast.NewNodeKind("FootnoteBackLink")
// KindFootnoteBacklink is a NodeKind of the FootnoteBacklink node.
var KindFootnoteBacklink = gast.NewNodeKind("FootnoteBacklink")
// Kind implements Node.Kind.
func (n *FootnoteBackLink) Kind() gast.NodeKind {
return KindFootnoteBackLink
func (n *FootnoteBacklink) Kind() gast.NodeKind {
return KindFootnoteBacklink
}
// NewFootnoteBackLink returns a new FootnoteBackLink node.
func NewFootnoteBackLink(index int) *FootnoteBackLink {
return &FootnoteBackLink{
Index: index,
// NewFootnoteBacklink returns a new FootnoteBacklink node.
func NewFootnoteBacklink(index int) *FootnoteBacklink {
return &FootnoteBacklink{
Index: index,
RefCount: 0,
}
}

View File

@@ -2,6 +2,8 @@ package extension
import (
"bytes"
"strconv"
"github.com/yuin/goldmark"
gast "github.com/yuin/goldmark/ast"
"github.com/yuin/goldmark/extension/ast"
@@ -10,10 +12,10 @@ import (
"github.com/yuin/goldmark/renderer/html"
"github.com/yuin/goldmark/text"
"github.com/yuin/goldmark/util"
"strconv"
)
var footnoteListKey = parser.NewContextKey()
var footnoteLinkListKey = parser.NewContextKey()
type footnoteBlockParser struct {
}
@@ -164,7 +166,20 @@ func (s *footnoteParser) Parse(parent gast.Node, block text.Reader, pc parser.Co
return nil
}
return ast.NewFootnoteLink(index)
fnlink := ast.NewFootnoteLink(index)
var fnlist []*ast.FootnoteLink
if tmp := pc.Get(footnoteLinkListKey); tmp != nil {
fnlist = tmp.([]*ast.FootnoteLink)
} else {
fnlist = []*ast.FootnoteLink{}
pc.Set(footnoteLinkListKey, fnlist)
}
pc.Set(footnoteLinkListKey, append(fnlist, fnlink))
if line[0] == '!' {
parent.AppendChild(parent, gast.NewTextSegment(text.NewSegment(segment.Start, segment.Start+1)))
}
return fnlink
}
type footnoteASTTransformer struct {
@@ -180,23 +195,46 @@ func NewFootnoteASTTransformer() parser.ASTTransformer {
func (a *footnoteASTTransformer) Transform(node *gast.Document, reader text.Reader, pc parser.Context) {
var list *ast.FootnoteList
if tlist := pc.Get(footnoteListKey); tlist != nil {
list = tlist.(*ast.FootnoteList)
} else {
var fnlist []*ast.FootnoteLink
if tmp := pc.Get(footnoteListKey); tmp != nil {
list = tmp.(*ast.FootnoteList)
}
if tmp := pc.Get(footnoteLinkListKey); tmp != nil {
fnlist = tmp.([]*ast.FootnoteLink)
}
pc.Set(footnoteListKey, nil)
pc.Set(footnoteLinkListKey, nil)
if list == nil {
return
}
pc.Set(footnoteListKey, nil)
counter := map[int]int{}
if fnlist != nil {
for _, fnlink := range fnlist {
if fnlink.Index >= 0 {
counter[fnlink.Index]++
}
}
for _, fnlink := range fnlist {
fnlink.RefCount = counter[fnlink.Index]
}
}
for footnote := list.FirstChild(); footnote != nil; {
var container gast.Node = footnote
next := footnote.NextSibling()
if fc := container.LastChild(); fc != nil && gast.IsParagraph(fc) {
container = fc
}
index := footnote.(*ast.Footnote).Index
fn := footnote.(*ast.Footnote)
index := fn.Index
if index < 0 {
list.RemoveChild(list, footnote)
} else {
container.AppendChild(container, ast.NewFootnoteBackLink(index))
backLink := ast.NewFootnoteBacklink(index)
backLink.RefCount = counter[index]
container.AppendChild(container, backLink)
}
footnote = next
}
@@ -214,19 +252,250 @@ func (a *footnoteASTTransformer) Transform(node *gast.Document, reader text.Read
node.AppendChild(node, list)
}
// FootnoteConfig holds configuration values for the footnote extension.
//
// Link* and Backlink* configurations have some variables:
// Occurrances of “^^” in the string will be replaced by the
// corresponding footnote number in the HTML output.
// Occurrances of “%%” will be replaced by a number for the
// reference (footnotes can have multiple references).
type FootnoteConfig struct {
html.Config
// IDPrefix is a prefix for the id attributes generated by footnotes.
IDPrefix []byte
// IDPrefix is a function that determines the id attribute for given Node.
IDPrefixFunction func(gast.Node) []byte
// LinkTitle is an optional title attribute for footnote links.
LinkTitle []byte
// BacklinkTitle is an optional title attribute for footnote backlinks.
BacklinkTitle []byte
// LinkClass is a class for footnote links.
LinkClass []byte
// BacklinkClass is a class for footnote backlinks.
BacklinkClass []byte
// BacklinkHTML is an HTML content for footnote backlinks.
BacklinkHTML []byte
}
// FootnoteOption interface is a functional option interface for the extension.
type FootnoteOption interface {
renderer.Option
// SetFootnoteOption sets given option to the extension.
SetFootnoteOption(*FootnoteConfig)
}
// NewFootnoteConfig returns a new Config with defaults.
func NewFootnoteConfig() FootnoteConfig {
return FootnoteConfig{
Config: html.NewConfig(),
LinkTitle: []byte(""),
BacklinkTitle: []byte(""),
LinkClass: []byte("footnote-ref"),
BacklinkClass: []byte("footnote-backref"),
BacklinkHTML: []byte("&#x21a9;&#xfe0e;"),
}
}
// SetOption implements renderer.SetOptioner.
func (c *FootnoteConfig) SetOption(name renderer.OptionName, value interface{}) {
switch name {
case optFootnoteIDPrefixFunction:
c.IDPrefixFunction = value.(func(gast.Node) []byte)
case optFootnoteIDPrefix:
c.IDPrefix = value.([]byte)
case optFootnoteLinkTitle:
c.LinkTitle = value.([]byte)
case optFootnoteBacklinkTitle:
c.BacklinkTitle = value.([]byte)
case optFootnoteLinkClass:
c.LinkClass = value.([]byte)
case optFootnoteBacklinkClass:
c.BacklinkClass = value.([]byte)
case optFootnoteBacklinkHTML:
c.BacklinkHTML = value.([]byte)
default:
c.Config.SetOption(name, value)
}
}
type withFootnoteHTMLOptions struct {
value []html.Option
}
func (o *withFootnoteHTMLOptions) SetConfig(c *renderer.Config) {
if o.value != nil {
for _, v := range o.value {
v.(renderer.Option).SetConfig(c)
}
}
}
func (o *withFootnoteHTMLOptions) SetFootnoteOption(c *FootnoteConfig) {
if o.value != nil {
for _, v := range o.value {
v.SetHTMLOption(&c.Config)
}
}
}
// WithFootnoteHTMLOptions is functional option that wraps goldmark HTMLRenderer options.
func WithFootnoteHTMLOptions(opts ...html.Option) FootnoteOption {
return &withFootnoteHTMLOptions{opts}
}
const optFootnoteIDPrefix renderer.OptionName = "FootnoteIDPrefix"
type withFootnoteIDPrefix struct {
value []byte
}
func (o *withFootnoteIDPrefix) SetConfig(c *renderer.Config) {
c.Options[optFootnoteIDPrefix] = o.value
}
func (o *withFootnoteIDPrefix) SetFootnoteOption(c *FootnoteConfig) {
c.IDPrefix = o.value
}
// WithFootnoteIDPrefix is a functional option that is a prefix for the id attributes generated by footnotes.
func WithFootnoteIDPrefix(a []byte) FootnoteOption {
return &withFootnoteIDPrefix{a}
}
const optFootnoteIDPrefixFunction renderer.OptionName = "FootnoteIDPrefixFunction"
type withFootnoteIDPrefixFunction struct {
value func(gast.Node) []byte
}
func (o *withFootnoteIDPrefixFunction) SetConfig(c *renderer.Config) {
c.Options[optFootnoteIDPrefixFunction] = o.value
}
func (o *withFootnoteIDPrefixFunction) SetFootnoteOption(c *FootnoteConfig) {
c.IDPrefixFunction = o.value
}
// WithFootnoteIDPrefixFunction is a functional option that is a prefix for the id attributes generated by footnotes.
func WithFootnoteIDPrefixFunction(a func(gast.Node) []byte) FootnoteOption {
return &withFootnoteIDPrefixFunction{a}
}
const optFootnoteLinkTitle renderer.OptionName = "FootnoteLinkTitle"
type withFootnoteLinkTitle struct {
value []byte
}
func (o *withFootnoteLinkTitle) SetConfig(c *renderer.Config) {
c.Options[optFootnoteLinkTitle] = o.value
}
func (o *withFootnoteLinkTitle) SetFootnoteOption(c *FootnoteConfig) {
c.LinkTitle = o.value
}
// WithFootnoteLinkTitle is a functional option that is an optional title attribute for footnote links.
func WithFootnoteLinkTitle(a []byte) FootnoteOption {
return &withFootnoteLinkTitle{a}
}
const optFootnoteBacklinkTitle renderer.OptionName = "FootnoteBacklinkTitle"
type withFootnoteBacklinkTitle struct {
value []byte
}
func (o *withFootnoteBacklinkTitle) SetConfig(c *renderer.Config) {
c.Options[optFootnoteBacklinkTitle] = o.value
}
func (o *withFootnoteBacklinkTitle) SetFootnoteOption(c *FootnoteConfig) {
c.BacklinkTitle = o.value
}
// WithFootnoteBacklinkTitle is a functional option that is an optional title attribute for footnote backlinks.
func WithFootnoteBacklinkTitle(a []byte) FootnoteOption {
return &withFootnoteBacklinkTitle{a}
}
const optFootnoteLinkClass renderer.OptionName = "FootnoteLinkClass"
type withFootnoteLinkClass struct {
value []byte
}
func (o *withFootnoteLinkClass) SetConfig(c *renderer.Config) {
c.Options[optFootnoteLinkClass] = o.value
}
func (o *withFootnoteLinkClass) SetFootnoteOption(c *FootnoteConfig) {
c.LinkClass = o.value
}
// WithFootnoteLinkClass is a functional option that is a class for footnote links.
func WithFootnoteLinkClass(a []byte) FootnoteOption {
return &withFootnoteLinkClass{a}
}
const optFootnoteBacklinkClass renderer.OptionName = "FootnoteBacklinkClass"
type withFootnoteBacklinkClass struct {
value []byte
}
func (o *withFootnoteBacklinkClass) SetConfig(c *renderer.Config) {
c.Options[optFootnoteBacklinkClass] = o.value
}
func (o *withFootnoteBacklinkClass) SetFootnoteOption(c *FootnoteConfig) {
c.BacklinkClass = o.value
}
// WithFootnoteBacklinkClass is a functional option that is a class for footnote backlinks.
func WithFootnoteBacklinkClass(a []byte) FootnoteOption {
return &withFootnoteBacklinkClass{a}
}
const optFootnoteBacklinkHTML renderer.OptionName = "FootnoteBacklinkHTML"
type withFootnoteBacklinkHTML struct {
value []byte
}
func (o *withFootnoteBacklinkHTML) SetConfig(c *renderer.Config) {
c.Options[optFootnoteBacklinkHTML] = o.value
}
func (o *withFootnoteBacklinkHTML) SetFootnoteOption(c *FootnoteConfig) {
c.BacklinkHTML = o.value
}
// WithFootnoteBacklinkHTML is an HTML content for footnote backlinks.
func WithFootnoteBacklinkHTML(a []byte) FootnoteOption {
return &withFootnoteBacklinkHTML{a}
}
// FootnoteHTMLRenderer is a renderer.NodeRenderer implementation that
// renders FootnoteLink nodes.
type FootnoteHTMLRenderer struct {
html.Config
FootnoteConfig
}
// NewFootnoteHTMLRenderer returns a new FootnoteHTMLRenderer.
func NewFootnoteHTMLRenderer(opts ...html.Option) renderer.NodeRenderer {
func NewFootnoteHTMLRenderer(opts ...FootnoteOption) renderer.NodeRenderer {
r := &FootnoteHTMLRenderer{
Config: html.NewConfig(),
FootnoteConfig: NewFootnoteConfig(),
}
for _, opt := range opts {
opt.SetHTMLOption(&r.Config)
opt.SetFootnoteOption(&r.FootnoteConfig)
}
return r
}
@@ -234,7 +503,7 @@ func NewFootnoteHTMLRenderer(opts ...html.Option) renderer.NodeRenderer {
// RegisterFuncs implements renderer.NodeRenderer.RegisterFuncs.
func (r *FootnoteHTMLRenderer) RegisterFuncs(reg renderer.NodeRendererFuncRegisterer) {
reg.Register(ast.KindFootnoteLink, r.renderFootnoteLink)
reg.Register(ast.KindFootnoteBackLink, r.renderFootnoteBackLink)
reg.Register(ast.KindFootnoteBacklink, r.renderFootnoteBacklink)
reg.Register(ast.KindFootnote, r.renderFootnote)
reg.Register(ast.KindFootnoteList, r.renderFootnoteList)
}
@@ -243,25 +512,45 @@ func (r *FootnoteHTMLRenderer) renderFootnoteLink(w util.BufWriter, source []byt
if entering {
n := node.(*ast.FootnoteLink)
is := strconv.Itoa(n.Index)
_, _ = w.WriteString(`<sup id="fnref:`)
_, _ = w.WriteString(`<sup id="`)
_, _ = w.Write(r.idPrefix(node))
_, _ = w.WriteString(`fnref:`)
_, _ = w.WriteString(is)
_, _ = w.WriteString(`"><a href="#fn:`)
_, _ = w.WriteString(`"><a href="#`)
_, _ = w.Write(r.idPrefix(node))
_, _ = w.WriteString(`fn:`)
_, _ = w.WriteString(is)
_, _ = w.WriteString(`" class="footnote-ref" role="doc-noteref">`)
_, _ = w.WriteString(`" class="`)
_, _ = w.Write(applyFootnoteTemplate(r.FootnoteConfig.LinkClass,
n.Index, n.RefCount))
if len(r.FootnoteConfig.LinkTitle) > 0 {
_, _ = w.WriteString(`" title="`)
_, _ = w.Write(util.EscapeHTML(applyFootnoteTemplate(r.FootnoteConfig.LinkTitle, n.Index, n.RefCount)))
}
_, _ = w.WriteString(`" role="doc-noteref">`)
_, _ = w.WriteString(is)
_, _ = w.WriteString(`</a></sup>`)
}
return gast.WalkContinue, nil
}
func (r *FootnoteHTMLRenderer) renderFootnoteBackLink(w util.BufWriter, source []byte, node gast.Node, entering bool) (gast.WalkStatus, error) {
func (r *FootnoteHTMLRenderer) renderFootnoteBacklink(w util.BufWriter, source []byte, node gast.Node, entering bool) (gast.WalkStatus, error) {
if entering {
n := node.(*ast.FootnoteBackLink)
n := node.(*ast.FootnoteBacklink)
is := strconv.Itoa(n.Index)
_, _ = w.WriteString(` <a href="#fnref:`)
_, _ = w.WriteString(` <a href="#`)
_, _ = w.Write(r.idPrefix(node))
_, _ = w.WriteString(`fnref:`)
_, _ = w.WriteString(is)
_, _ = w.WriteString(`" class="footnote-backref" role="doc-backlink">`)
_, _ = w.WriteString("&#x21a9;&#xfe0e;")
_, _ = w.WriteString(`" class="`)
_, _ = w.Write(applyFootnoteTemplate(r.FootnoteConfig.BacklinkClass, n.Index, n.RefCount))
if len(r.FootnoteConfig.BacklinkTitle) > 0 {
_, _ = w.WriteString(`" title="`)
_, _ = w.Write(util.EscapeHTML(applyFootnoteTemplate(r.FootnoteConfig.BacklinkTitle, n.Index, n.RefCount)))
}
_, _ = w.WriteString(`" role="doc-backlink">`)
_, _ = w.Write(applyFootnoteTemplate(r.FootnoteConfig.BacklinkHTML, n.Index, n.RefCount))
_, _ = w.WriteString(`</a>`)
}
return gast.WalkContinue, nil
@@ -271,7 +560,9 @@ func (r *FootnoteHTMLRenderer) renderFootnote(w util.BufWriter, source []byte, n
n := node.(*ast.Footnote)
is := strconv.Itoa(n.Index)
if entering {
_, _ = w.WriteString(`<li id="fn:`)
_, _ = w.WriteString(`<li id="`)
_, _ = w.Write(r.idPrefix(node))
_, _ = w.WriteString(`fn:`)
_, _ = w.WriteString(is)
_, _ = w.WriteString(`" role="doc-endnote"`)
if node.Attributes() != nil {
@@ -312,11 +603,54 @@ func (r *FootnoteHTMLRenderer) renderFootnoteList(w util.BufWriter, source []byt
return gast.WalkContinue, nil
}
func (r *FootnoteHTMLRenderer) idPrefix(node gast.Node) []byte {
if r.FootnoteConfig.IDPrefix != nil {
return r.FootnoteConfig.IDPrefix
}
if r.FootnoteConfig.IDPrefixFunction != nil {
return r.FootnoteConfig.IDPrefixFunction(node)
}
return []byte("")
}
func applyFootnoteTemplate(b []byte, index, refCount int) []byte {
fast := true
for i, c := range b {
if i != 0 {
if b[i-1] == '^' && c == '^' {
fast = false
break
}
if b[i-1] == '%' && c == '%' {
fast = false
break
}
}
}
if fast {
return b
}
is := []byte(strconv.Itoa(index))
rs := []byte(strconv.Itoa(refCount))
ret := bytes.Replace(b, []byte("^^"), is, -1)
return bytes.Replace(ret, []byte("%%"), rs, -1)
}
type footnote struct {
options []FootnoteOption
}
// Footnote is an extension that allow you to use PHP Markdown Extra Footnotes.
var Footnote = &footnote{}
var Footnote = &footnote{
options: []FootnoteOption{},
}
// NewFootnote returns a new extension with given options.
func NewFootnote(opts ...FootnoteOption) goldmark.Extender {
return &footnote{
options: opts,
}
}
func (e *footnote) Extend(m goldmark.Markdown) {
m.Parser().AddOptions(
@@ -331,6 +665,6 @@ func (e *footnote) Extend(m goldmark.Markdown) {
),
)
m.Renderer().AddOptions(renderer.WithNodeRenderers(
util.Prioritized(NewFootnoteHTMLRenderer(), 500),
util.Prioritized(NewFootnoteHTMLRenderer(e.options...), 500),
))
}

View File

@@ -11,9 +11,9 @@ import (
"github.com/yuin/goldmark/util"
)
var wwwURLRegxp = regexp.MustCompile(`^www\.[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]+(?:(?:/|[#?])[-a-zA-Z0-9@:%_\+.~#!?&//=\(\);,'">\^{}\[\]` + "`" + `]*)?`)
var wwwURLRegxp = regexp.MustCompile(`^www\.[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-z]+(?:[/#?][-a-zA-Z0-9@:%_\+.~#!?&/=\(\);,'">\^{}\[\]` + "`" + `]*)?`)
var urlRegexp = regexp.MustCompile(`^(?:http|https|ftp):\/\/(?:www\.)?[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]+(?:(?:/|[#?])[-a-zA-Z0-9@:%_+.~#$!?&//=\(\);,'">\^{}\[\]` + "`" + `]*)?`)
var urlRegexp = regexp.MustCompile(`^(?:http|https|ftp)://[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-z]+(?::\d+)?(?:[/#?][-a-zA-Z0-9@:%_+.~#$!?&/=\(\);,'">\^{}\[\]` + "`" + `]*)?`)
// An LinkifyConfig struct is a data structure that holds configuration of the
// Linkify extension.
@@ -24,10 +24,12 @@ type LinkifyConfig struct {
EmailRegexp *regexp.Regexp
}
const optLinkifyAllowedProtocols parser.OptionName = "LinkifyAllowedProtocols"
const optLinkifyURLRegexp parser.OptionName = "LinkifyURLRegexp"
const optLinkifyWWWRegexp parser.OptionName = "LinkifyWWWRegexp"
const optLinkifyEmailRegexp parser.OptionName = "LinkifyEmailRegexp"
const (
optLinkifyAllowedProtocols parser.OptionName = "LinkifyAllowedProtocols"
optLinkifyURLRegexp parser.OptionName = "LinkifyURLRegexp"
optLinkifyWWWRegexp parser.OptionName = "LinkifyWWWRegexp"
optLinkifyEmailRegexp parser.OptionName = "LinkifyEmailRegexp"
)
// SetOption implements SetOptioner.
func (c *LinkifyConfig) SetOption(name parser.OptionName, value interface{}) {
@@ -156,10 +158,12 @@ func (s *linkifyParser) Trigger() []byte {
return []byte{' ', '*', '_', '~', '('}
}
var protoHTTP = []byte("http:")
var protoHTTPS = []byte("https:")
var protoFTP = []byte("ftp:")
var domainWWW = []byte("www.")
var (
protoHTTP = []byte("http:")
protoHTTPS = []byte("https:")
protoFTP = []byte("ftp:")
domainWWW = []byte("www.")
)
func (s *linkifyParser) Parse(parent ast.Node, block text.Reader, pc parser.Context) ast.Node {
if pc.IsInLinkLabel() {

View File

@@ -15,6 +15,14 @@ import (
"github.com/yuin/goldmark/util"
)
var escapedPipeCellListKey = parser.NewContextKey()
type escapedPipeCell struct {
Cell *ast.TableCell
Pos []int
Transformed bool
}
// TableCellAlignMethod indicates how are table cells aligned in HTML format.indicates how are table cells aligned in HTML format.
type TableCellAlignMethod int
@@ -148,7 +156,7 @@ func (b *tableParagraphTransformer) Transform(node *gast.Paragraph, reader text.
if alignments == nil {
continue
}
header := b.parseRow(lines.At(i-1), alignments, true, reader)
header := b.parseRow(lines.At(i-1), alignments, true, reader, pc)
if header == nil || len(alignments) != header.ChildCount() {
return
}
@@ -156,7 +164,7 @@ func (b *tableParagraphTransformer) Transform(node *gast.Paragraph, reader text.
table.Alignments = alignments
table.AppendChild(table, ast.NewTableHeader(header))
for j := i + 1; j < lines.Len(); j++ {
table.AppendChild(table, b.parseRow(lines.At(j), alignments, false, reader))
table.AppendChild(table, b.parseRow(lines.At(j), alignments, false, reader, pc))
}
node.Lines().SetSliced(0, i-1)
node.Parent().InsertAfter(node.Parent(), node, table)
@@ -170,7 +178,7 @@ func (b *tableParagraphTransformer) Transform(node *gast.Paragraph, reader text.
}
}
func (b *tableParagraphTransformer) parseRow(segment text.Segment, alignments []ast.Alignment, isHeader bool, reader text.Reader) *ast.TableRow {
func (b *tableParagraphTransformer) parseRow(segment text.Segment, alignments []ast.Alignment, isHeader bool, reader text.Reader, pc parser.Context) *ast.TableRow {
source := reader.Source()
line := segment.Value(source)
pos := 0
@@ -194,18 +202,39 @@ func (b *tableParagraphTransformer) parseRow(segment text.Segment, alignments []
} else {
alignment = alignments[i]
}
closure := util.FindClosure(line[pos:], byte(0), '|', true, false)
if closure < 0 {
closure = len(line[pos:])
}
var escapedCell *escapedPipeCell
node := ast.NewTableCell()
seg := text.NewSegment(segment.Start+pos, segment.Start+pos+closure)
node.Alignment = alignment
hasBacktick := false
closure := pos
for ; closure < limit; closure++ {
if line[closure] == '`' {
hasBacktick = true
}
if line[closure] == '|' {
if closure == 0 || line[closure-1] != '\\' {
break
} else if hasBacktick {
if escapedCell == nil {
escapedCell = &escapedPipeCell{node, []int{}, false}
escapedList := pc.ComputeIfAbsent(escapedPipeCellListKey,
func() interface{} {
return []*escapedPipeCell{}
}).([]*escapedPipeCell)
escapedList = append(escapedList, escapedCell)
pc.Set(escapedPipeCellListKey, escapedList)
}
escapedCell.Pos = append(escapedCell.Pos, segment.Start+closure-1)
}
}
}
seg := text.NewSegment(segment.Start+pos, segment.Start+closure)
seg = seg.TrimLeftSpace(source)
seg = seg.TrimRightSpace(source)
node.Lines().Append(seg)
node.Alignment = alignment
row.AppendChild(row, node)
pos += closure + 1
pos = closure + 1
}
for ; i < len(alignments); i++ {
row.AppendChild(row, ast.NewTableCell())
@@ -243,6 +272,61 @@ func (b *tableParagraphTransformer) parseDelimiter(segment text.Segment, reader
return alignments
}
type tableASTTransformer struct {
}
var defaultTableASTTransformer = &tableASTTransformer{}
// NewTableASTTransformer returns a parser.ASTTransformer for tables.
func NewTableASTTransformer() parser.ASTTransformer {
return defaultTableASTTransformer
}
func (a *tableASTTransformer) Transform(node *gast.Document, reader text.Reader, pc parser.Context) {
lst := pc.Get(escapedPipeCellListKey)
if lst == nil {
return
}
pc.Set(escapedPipeCellListKey, nil)
for _, v := range lst.([]*escapedPipeCell) {
if v.Transformed {
continue
}
_ = gast.Walk(v.Cell, func(n gast.Node, entering bool) (gast.WalkStatus, error) {
if !entering || n.Kind() != gast.KindCodeSpan {
return gast.WalkContinue, nil
}
for c := n.FirstChild(); c != nil; {
next := c.NextSibling()
if c.Kind() != gast.KindText {
c = next
continue
}
parent := c.Parent()
ts := &c.(*gast.Text).Segment
n := c
for _, v := range lst.([]*escapedPipeCell) {
for _, pos := range v.Pos {
if ts.Start <= pos && pos < ts.Stop {
segment := n.(*gast.Text).Segment
n1 := gast.NewRawTextSegment(segment.WithStop(pos))
n2 := gast.NewRawTextSegment(segment.WithStart(pos + 1))
parent.InsertAfter(parent, n, n1)
parent.InsertAfter(parent, n1, n2)
parent.RemoveChild(parent, n)
n = n2
v.Transformed = true
}
}
}
c = next
}
return gast.WalkContinue, nil
})
}
}
// TableHTMLRenderer is a renderer.NodeRenderer implementation that
// renders Table nodes.
type TableHTMLRenderer struct {
@@ -419,7 +503,7 @@ func (r *TableHTMLRenderer) renderTableCell(w util.BufWriter, source []byte, nod
cob.AppendByte(';')
}
style := fmt.Sprintf("text-align:%s", n.Alignment.String())
cob.Append(util.StringToReadOnlyBytes(style))
cob.AppendString(style)
n.SetAttributeString("style", cob.Bytes())
}
}
@@ -454,9 +538,14 @@ func NewTable(opts ...TableOption) goldmark.Extender {
}
func (e *table) Extend(m goldmark.Markdown) {
m.Parser().AddOptions(parser.WithParagraphTransformers(
util.Prioritized(NewTableParagraphTransformer(), 200),
))
m.Parser().AddOptions(
parser.WithParagraphTransformers(
util.Prioritized(NewTableParagraphTransformer(), 200),
),
parser.WithASTTransformers(
util.Prioritized(defaultTableASTTransformer, 0),
),
)
m.Renderer().AddOptions(renderer.WithNodeRenderers(
util.Prioritized(NewTableHTMLRenderer(e.options...), 500),
))

View File

@@ -1,3 +1,3 @@
module github.com/yuin/goldmark
go 1.13
go 1.15

View File

@@ -49,6 +49,12 @@ func (b *codeBlockParser) Continue(node ast.Node, reader text.Reader, pc Context
}
reader.AdvanceAndSetPadding(pos, padding)
_, segment = reader.PeekLine()
// if code block line starts with a tab, keep a tab as it is.
if segment.Padding != 0 {
preserveLeadingTabInCodeBlock(&segment, reader)
}
node.Lines().Append(segment)
reader.Advance(segment.Len() - 1)
return Continue | NoChildren
@@ -77,3 +83,14 @@ func (b *codeBlockParser) CanInterruptParagraph() bool {
func (b *codeBlockParser) CanAcceptIndentedLine() bool {
return true
}
func preserveLeadingTabInCodeBlock(segment *text.Segment, reader text.Reader) {
offsetWithPadding := reader.LineOffset()
sl, ss := reader.Position()
reader.SetPosition(sl, text.NewSegment(ss.Start-1, ss.Stop))
if offsetWithPadding == reader.LineOffset() {
segment.Padding = 0
segment.Start--
}
reader.SetPosition(sl, ss)
}

View File

@@ -71,6 +71,10 @@ func (b *fencedCodeBlockParser) Open(parent ast.Node, reader text.Reader, pc Con
func (b *fencedCodeBlockParser) Continue(node ast.Node, reader text.Reader, pc Context) State {
line, segment := reader.PeekLine()
fdata := pc.Get(fencedCodeBlockInfoKey).(*fenceData)
// if code block line starts with a tab, keep a tab as it is.
if segment.Padding != 0 {
preserveLeadingTabInCodeBlock(&segment, reader)
}
w, pos := util.IndentWidth(line, reader.LineOffset())
if w < 4 {
i := pos

View File

@@ -2,7 +2,6 @@ package parser
import (
"fmt"
"regexp"
"strings"
"github.com/yuin/goldmark/ast"
@@ -113,8 +112,6 @@ func (s *linkParser) Trigger() []byte {
return []byte{'!', '[', ']'}
}
var linkDestinationRegexp = regexp.MustCompile(`\s*([^\s].+)`)
var linkTitleRegexp = regexp.MustCompile(`\s+(\)|["'\(].+)`)
var linkBottom = NewContextKey()
func (s *linkParser) Parse(parent ast.Node, block text.Reader, pc Context) ast.Node {
@@ -293,20 +290,17 @@ func (s *linkParser) parseLink(parent ast.Node, last *linkLabelState, block text
func parseLinkDestination(block text.Reader) ([]byte, bool) {
block.SkipSpaces()
line, _ := block.PeekLine()
buf := []byte{}
if block.Peek() == '<' {
i := 1
for i < len(line) {
c := line[i]
if c == '\\' && i < len(line)-1 && util.IsPunct(line[i+1]) {
buf = append(buf, '\\', line[i+1])
i += 2
continue
} else if c == '>' {
block.Advance(i + 1)
return line[1:i], true
}
buf = append(buf, c)
i++
}
return nil, false
@@ -316,7 +310,6 @@ func parseLinkDestination(block text.Reader) ([]byte, bool) {
for i < len(line) {
c := line[i]
if c == '\\' && i < len(line)-1 && util.IsPunct(line[i+1]) {
buf = append(buf, '\\', line[i+1])
i += 2
continue
} else if c == '(' {
@@ -329,7 +322,6 @@ func parseLinkDestination(block text.Reader) ([]byte, bool) {
} else if util.IsSpace(c) {
break
}
buf = append(buf, c)
i++
}
block.Advance(i)

View File

@@ -138,6 +138,9 @@ type Context interface {
// Get returns a value associated with the given key.
Get(ContextKey) interface{}
// ComputeIfAbsent computes a value if a value associated with the given key is absent and returns the value.
ComputeIfAbsent(ContextKey, func() interface{}) interface{}
// Set sets the given value to the context.
Set(ContextKey, interface{})
@@ -252,6 +255,15 @@ func (p *parseContext) Get(key ContextKey) interface{} {
return p.store[key]
}
func (p *parseContext) ComputeIfAbsent(key ContextKey, f func() interface{}) interface{} {
v := p.store[key]
if v == nil {
v = f()
p.store[key] = v
}
return v
}
func (p *parseContext) Set(key ContextKey, value interface{}) {
p.store[key] = value
}

View File

@@ -2,10 +2,11 @@ package parser
import (
"bytes"
"regexp"
"github.com/yuin/goldmark/ast"
"github.com/yuin/goldmark/text"
"github.com/yuin/goldmark/util"
"regexp"
)
type rawHTMLParser struct {
@@ -67,8 +68,6 @@ func (s *rawHTMLParser) parseSingleLineRegexp(reg *regexp.Regexp, block text.Rea
return node
}
var dummyMatch = [][]byte{}
func (s *rawHTMLParser) parseMultiLineRegexp(reg *regexp.Regexp, block text.Reader, pc Context) ast.Node {
sline, ssegment := block.Position()
if block.Match(reg) {
@@ -102,7 +101,3 @@ func (s *rawHTMLParser) parseMultiLineRegexp(reg *regexp.Regexp, block text.Read
}
return nil
}
func (s *rawHTMLParser) CloseBlock(parent ast.Node, pc Context) {
// nothing to do
}

View File

@@ -37,6 +37,12 @@ func (b *CopyOnWriteBuffer) Write(value []byte) {
b.buffer = append(b.buffer, value...)
}
// WriteString writes given string to the buffer.
// WriteString allocate new buffer and clears it at the first time.
func (b *CopyOnWriteBuffer) WriteString(value string) {
b.Write(StringToReadOnlyBytes(value))
}
// Append appends given bytes to the buffer.
// Append copy buffer at the first time.
func (b *CopyOnWriteBuffer) Append(value []byte) {
@@ -49,6 +55,12 @@ func (b *CopyOnWriteBuffer) Append(value []byte) {
b.buffer = append(b.buffer, value...)
}
// AppendString appends given string to the buffer.
// AppendString copy buffer at the first time.
func (b *CopyOnWriteBuffer) AppendString(value string) {
b.Append(StringToReadOnlyBytes(value))
}
// WriteByte writes the given byte to the buffer.
// WriteByte allocate new buffer and clears it at the first time.
func (b *CopyOnWriteBuffer) WriteByte(c byte) {
@@ -804,7 +816,7 @@ func IsPunct(c byte) bool {
return punctTable[c] == 1
}
// IsPunct returns true if the given rune is a punctuation, otherwise false.
// IsPunctRune returns true if the given rune is a punctuation, otherwise false.
func IsPunctRune(r rune) bool {
return int32(r) <= 256 && IsPunct(byte(r)) || unicode.IsPunct(r)
}
@@ -814,7 +826,7 @@ func IsSpace(c byte) bool {
return spaceTable[c] == 1
}
// IsSpace returns true if the given rune is a space, otherwise false.
// IsSpaceRune returns true if the given rune is a space, otherwise false.
func IsSpaceRune(r rune) bool {
return int32(r) <= 256 && IsSpace(byte(r)) || unicode.IsSpace(r)
}

2
vendor/modules.txt vendored
View File

@@ -745,7 +745,7 @@ github.com/xi2/xz
# github.com/yohcop/openid-go v1.0.0
## explicit
github.com/yohcop/openid-go
# github.com/yuin/goldmark v1.2.1
# github.com/yuin/goldmark v1.3.3
## explicit
github.com/yuin/goldmark
github.com/yuin/goldmark/ast

View File

@@ -3592,18 +3592,21 @@ function initIssueList() {
fullTextSearch: true
});
function excludeLabel (item) {
const href = $(item).attr('href');
const id = $(item).data('label-id');
const regStr = `labels=((?:-?[0-9]+%2c)*)(${id})((?:%2c-?[0-9]+)*)&`;
const newStr = 'labels=$1-$2$3&';
window.location = href.replace(new RegExp(regStr), newStr);
}
$('.menu a.label-filter-item').each(function () {
$(this).on('click', function (e) {
if (e.altKey) {
e.preventDefault();
const href = $(this).attr('href');
const id = $(this).data('label-id');
const regStr = `labels=(-?[0-9]+%2c)*(${id})(%2c-?[0-9]+)*&`;
const newStr = 'labels=$1-$2$3&';
window.location = href.replace(new RegExp(regStr), newStr);
excludeLabel(this);
}
});
});
@@ -3611,17 +3614,8 @@ function initIssueList() {
$('.menu .ui.dropdown.label-filter').on('keydown', (e) => {
if (e.altKey && e.keyCode === 13) {
const selectedItems = $('.menu .ui.dropdown.label-filter .menu .item.selected');
if (selectedItems.length > 0) {
const item = $(selectedItems[0]);
const href = item.attr('href');
const id = item.data('label-id');
const regStr = `labels=(-?[0-9]+%2c)*(${id})(%2c-?[0-9]+)*&`;
const newStr = 'labels=$1-$2$3&';
window.location = href.replace(new RegExp(regStr), newStr);
excludeLabel($(selectedItems[0]));
}
}
});

View File

@@ -5,7 +5,7 @@ const headingSelector = '.markdown h1, .markdown h2, .markdown h3, .markdown h4,
function scrollToAnchor() {
if (document.querySelector(':target')) return;
if (!window.location.hash || window.location.hash.length <= 1) return;
const id = window.location.hash.substring(1);
const id = decodeURIComponent(window.location.hash.substring(1));
const el = document.getElementById(`user-content-${id}`);
if (el) {
el.scrollIntoView();

View File

@@ -1320,6 +1320,7 @@
padding-top: 5px;
padding-right: 10px;
color: #000000;
font-size: 1.5rem;
&:hover {
color: #4078c0;