Add contributors view in repo sidebar

[bircni]

Adds a contributors view as GitHub has in the repo sidebar

[silverwind]

I think you can use existing avatar template helpers, same as should be used in issue list or elsewhere.

Also I recommend setting a explicit size=32 parameter on it to make the backend return downsized avatars.

[silverwind]

This rule should removed and it should use global avatar styles (if they exist). And we don't do fully-round avatars (design choice).

[silverwind]

Does it have performance impact?

[wxiaoguang]

Sorry, you will know that the design is not feasible if you look into the details of GetContributorStats, especially on a heavy server which has a lot of large repos.

Although GitHub shows the contributors, I am sure it uses a much better approach to avoid performance problem.

[a1012112796]

Yes, at lest should lazy loading like previous usage.

gitea/web_src/js/components/RepoContributors.vue

Lines 134 to 149 in 6f9fa55

	async fetchGraphData() {
	this.isLoading = true;
	try {
	let response: Response;
	do {
	response = await GET(`${this.repoLink}/activity/contributors/data`);
	if (response.status === 202) {
	await sleep(1000); // wait for 1 second before retrying
	}
	} while (response.status === 202);
	if (response.ok) {
	const data = await response.json() as ContributorsData;
	const {total, ...other} = data;
	// below line might be deleted if we are sure go produces map always sorted by keys
	total.weeks = Object.fromEntries(Object.entries(total.weeks).sort());

[silverwind]

lazy loading

That's one solution but I think it's not ideal because of content pop-in. A ideal solution would be the have some server-side cache. I think the contributor stats already have some form of cache, maybe it can even be re-used, this user list can be built off the same data (top contributors sorted by contributions).

[wxiaoguang]

No, the problem is more complex than a simple cache.

I don't know whether you have benchmarked the related function. On my side, GetContributorStats takes about 20 seconds on gitea's git repo.

So, suppose you have 30 gitea-like (or even larger) git repositories, and these repositories are viewed frequently (say, a large server), then GetContributorStats will keep running the git command and take up 100% CPU for all time.

[silverwind]

Could be made an intelligent cache. On repo page with stale cache, run in background and don't show it initially, only when available and non-stale. On stats page run it synchronously via fetch (long loading state, same as GitHub).

That's at least how it works on GitHub. Another option is to have a scheduled background cron job, but that'll be more work overall for the server as it has to do it for all repos regardless whether they are being visited.

[wxiaoguang]

If let me guess, I would say GitHub uses a database-based incremental "contributor counter". If any push always triggers the re-calculation of the "contributor stats", it is unlikely a feasible solution for a large repo like Linux kernel.

[TheFox0x7]

Actually isn't that function racy? If the generation takes up a bit (like the 20 seconds you mentioned) wouldn't few users hitting it create multiple git processes as each is trying to fill the cache that does not exist yet?

If any push always triggers the re-calculation of the "contributor stats", it is unlikely a feasible solution for a large repo like Linux kernel.

Yeah it takes on my HW about 4 minutes to run getExtendedCommitStats git command and I've limited it to be from 2022 in the first place.

I think push could just append to the database table, but then it's an issue with force-pushes afaik (plus memory issue I've ran into while looking at secret scanning). I haven't thought about this in a while though.

[a1012112796]

So, suppose you have 30 gitea-like (or even larger) git repositories, and these repositories are viewed frequently (say, a large server), then GetContributorStats will keep running the git command and take up 100% CPU for all time.

so, looks it's an exist issue? How about add a queue to do generateContributorStats?

[silverwind]

If let me guess, I would say GitHub uses a database-based incremental "contributor counter".

Can't be. Visiting <repo>/graphs/contributors often loads many minutes for me on some repos. I think it's a cache like I described.

[wxiaoguang]

So, suppose you have 30 gitea-like (or even larger) git repositories, and these repositories are viewed frequently (say, a large server), then GetContributorStats will keep running the git command and take up 100% CPU for all time.

so, looks it's an exist issue? How about add a queue to do generateContributorStats?

The problem is there, but it doesn't really cause issue at the moment (unless some malicious users try to DoS a server).

At the moment, "GetContributorStats" is only called by fetch in the "Activity" sub-page, not the repo's home page.

I don't see a simple "queue" could really help the problem either.

[wxiaoguang]

Actually isn't that function racy? If the generation takes up a bit (like the 20 seconds you mentioned) wouldn't few users hitting it create multiple git processes as each is trying to fill the cache that does not exist yet?

There is also a lock in GetContributorStats

[TheFox0x7]

Actually isn't that function racy? If the generation takes up a bit (like the 20 seconds you mentioned) wouldn't few users hitting it create multiple git processes as each is trying to fill the cache that does not exist yet?

There is also a lock in GetContributorStats

Indeed there is and I missed it...