- Go 84.6%
- HTML 7.1%
- JavaScript 5.7%
- Shell 1.8%
- CSS 0.7%
946e2b83a1There is a danger of deadlock wherever we have database operations
inside a database iterator. The iterator itself pins a connection, so
the nested operations need another connection; if we've reached
maxOpenConns then it blocks until a connection is released. If all
connections are consumed by such iterators then none can make progress.
Luckily, we have limited concurrency for most such iterator loops. They
are part of scanning, pulling, reverting, etc where there is only ever
one such routine per folder. The exception is block reuse in the copier
routine, which is limited by the `Copiers` setting per folder. This
could in practice deadlock since you could set copiers to eight and end
up with six `AllLocalBlocksWithHash` iterators when `maxDBConns=6`, all
of which need to make additional database calls inside the loop.
This PR fixes the problem twice;
- The problematic loop does not need to be reentrant. The set of blocks
that may be returned by the iterator is finite so we can easily just
collect them to a slice before we start processing them. This avoids the
problem entirely.
- We do not need to limit database connections as strictly as we
currently do. Increase the maximum allowed, while reducing the number of
held-open idle connections slightly. This is not an exact science, but
ideally we want "most" operations to be able to use the pinned
connections to avoid cache churn. Most operations are short lived
queries, or single-goroutine iterators with short lived queries inside,
so four connections seems like it should usually be enough. 🤷
- Additionally, set a cap on Copiers. Currently you could set it to an
arbitrarily large number, which is not advantageous. Limit it to
2*NumCPU which scales somewhat with system performance.
Signed-off-by: Jakob Borg <jakob@kastelo.net>
|
||
|---|---|---|
| .github | ||
| assets | ||
| cmd | ||
| etc | ||
| gui | ||
| internal | ||
| lib | ||
| man | ||
| meta | ||
| proto | ||
| relnotes | ||
| script | ||
| test | ||
| .codecov.yml | ||
| .deepsource.toml | ||
| .gitattributes | ||
| .gitignore | ||
| .golangci.yml | ||
| .policy.yml | ||
| .yamlfmt | ||
| AUTHORS | ||
| buf.gen.yaml | ||
| buf.yaml | ||
| build.go | ||
| build.ps1 | ||
| build.sh | ||
| compat.yaml | ||
| CONDUCT.md | ||
| CONTRIBUTING.md | ||
| Dockerfile | ||
| Dockerfile.builder | ||
| Dockerfile.stcrashreceiver | ||
| Dockerfile.stdiscosrv | ||
| Dockerfile.strelaypoolsrv | ||
| Dockerfile.strelaysrv | ||
| Dockerfile.stupgrades | ||
| Dockerfile.ursrv | ||
| go.mod | ||
| go.sum | ||
| GOALS.md | ||
| LICENSE | ||
| README-Docker.md | ||
| README.md | ||
Goals
Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. We strive to fulfill the goals below. The goals are listed in order of importance, the most important ones first. This is the summary version of the goal list - for more commentary, see the full Goals document.
Syncthing should be:
-
Safe From Data Loss
Protecting the user's data is paramount. We take every reasonable precaution to avoid corrupting the user's files.
-
Secure Against Attackers
Again, protecting the user's data is paramount. Regardless of our other goals, we must never allow the user's data to be susceptible to eavesdropping or modification by unauthorized parties.
-
Easy to Use
Syncthing should be approachable, understandable, and inclusive.
-
Automatic
User interaction should be required only when absolutely necessary.
-
Universally Available
Syncthing should run on every common computer. We are mindful that the latest technology is not always available to every individual.
-
For Individuals
Syncthing is primarily about empowering the individual user with safe, secure, and easy to use file synchronization.
-
Everything Else
There are many things we care about that don't make it on to the list. It is fine to optimize for these values, as long as they are not in conflict with the stated goals above.
Getting Started
Take a look at the getting started guide.
There are a few examples for keeping Syncthing running in the background on your system in the etc directory. There are also several GUI implementations for Windows, Mac, and Linux.
Docker
To run Syncthing in Docker, see the Docker README.
Getting in Touch
The first and best point of contact is the Forum. If you've found something that is clearly a bug, feel free to report it in the GitHub issue tracker.
If you believe that you’ve found a Syncthing-related security vulnerability, please report it by emailing security@syncthing.net. Do not report it in the Forum or issue tracker.
Building
Building Syncthing from source is easy. After extracting the source bundle from
a release or checking out git, you just need to run go run build.go and the
binaries are created in ./bin. There's a guide with more details on the
build process.
Signed Releases
Release binaries are GPG signed with the key available from https://syncthing.net/security/. There is also a built-in automatic upgrade mechanism (disabled in some distribution channels) which uses a compiled in ECDSA signature. macOS and Windows binaries are also code-signed.
Documentation
Please see the Syncthing documentation site [source].
All code is licensed under the MPLv2 License.