Close our SQLAlchemy sessions

This is a temporary workaround to stop us leaving sessions around
until I write a decorator function

Signed-off-by: Mark Hymers <mhy@debian.org>

Implement and document the different names

Signed-off-by: Mark Hymers <mhy@debian.org>

Hack to not end up with None versions

Signed-off-by: Mark Hymers <mhy@debian.org>

Find suite by any of the random name types we have

Signed-off-by: Mark Hymers <mhy@debian.org>

Add database update to contain release_suite field

Signed-off-by: Mark Hymers <mhy@debian.org>

Add suite function

Signed-off-by: Mark Hymers <mhy@debian.org>

Improve suites() documentation

Signed-off-by: Mark Hymers <mhy@debian.org>

Improve dsc_in_suite function documentation

Signed-off-by: Mark Hymers <mhy@debian.org>

Improve archives() function documentation

Signed-off-by: Mark Hymers <mhy@debian.org>

Add initial suites and archives routines

Signed-off-by: Mark Hymers <mhy@debian.org>

Return filesize and sha256sum too

Signed-off-by: Mark Hymers <mhy@debian.org>

Run WSGI script from apache

Signed-off-by: Mark Hymers <mhy@debian.org>

First pass at a hacky mod_wsgi version

Signed-off-by: Mark Hymers <mhy@debian.org>

Add initial api.ftp-master.d.o apache config

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge in the initial web API code

Remove debugging code

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote-tracking branch 'origin/dakweb'

Merge remote-tracking branch 'mhy/master'

Lock {src,bin}_associations to stop other things writing to them during gps

Signed-off-by: Mark Hymers <mhy@debian.org>

Update install instructions

Signed-off-by: Mark Hymers <mhy@debian.org>

Create some directories needed for new dak installs

Signed-off-by: Mark Hymers <mhy@debian.org>

Call dak admin s-c in example install instructions

Signed-off-by: Mark Hymers <mhy@debian.org>

Reject uploads with Multi-Arch: no

Multi-Arch: no currently breaks wanna-build. We don't want that to happen.

Reference: https://bugs.debian.org/768353

Do not require arch:all packages in experimental uploads

This is to allow setting up an experimental arch:all buildd and will
be reverted later.

daklib/checks.py: check for Section field in binary packages

daklib/gpg.py: Ignore POLICY_URL keyword.

daklib/command.py: Add Cc option.

README.stable-point-release: Update for 7.7

Add copy & paste snippets for propups.

security NEW processing: add dak admin forget-signature

Handle packages with overrides in multiple components

This should make moving packages between components (e.g. packages
moving from non-free to main) less painful.

Bug: https://bugs.debian.org/690444

daklib/checks.py: check timestamp of .changes signature

This allows to eventually drop old entries from the signature_history
table.

dak make-pkg-file-mapping: do not include sources from other archives

Include only information about a source package's files if its .dsc is
in the requested archive.

Previously when xy_1.0-1 was in the requested archive and xy_1.0-2
using the same upstream tarball in a different archive, dak would
still include information about xy_1.0-2's upstream tarball resulting
in confusing entries on snapshot.d.o.

Reported-by: Salvatore Bonaccorso <carnil@debian.org>

dak make-pkg-file-mapping: ensure stable order

Include component and filename in the ORDER BY clause to ensure the
output order does not change.

test_gpg.py: test PGP messages and plaintext

test_gpg.py: also check signature timestamp

Add first unit tests for GnuPG wrapper.

dm-migrate: list source packages for which permissions were migrated.

cron.unchecked: drop unused do_dists function.

cron.unchecked: stop bothering with override files.

Merge remote-tracking branch 'paultag/paultag/bugfix/import-errno'

Import errno, used in Exception handling.

From a reject exception:

|   File "/srv/ftp-master.debian.org/dak/dak/daklib/upload.py", line 155, in check
|     if e.errno == errno.ENOENT:
| NameError: global name 'errno' is not defined

HashedFile: Raise FileDoesNotExist if referring to a non-existing file

A custom exception is nicer to handle in the caller.

daklib/upload.py: Use common base class for all exception.

Merge branch 'master' of /srv/ftp.debian.org/git/dak

Add ppc64el

Signed-off-by: Mark Hymers <mhy@debian.org>

cron.daily: Don't allow ssh trigger to run arbitrary SQL

Remove apache.conf-incoming.

incoming.d.o is no longer hosted on ftp-master.d.o.

Show last updated time on i.d.o

Signed-off-by: Mark Hymers <mhy@debian.org>

Remove old (and slow) code

Signed-off-by: Mark Hymers <mhy@debian.org>

Remove code working around an old issue

Signed-off-by: Mark Hymers <mhy@debian.org>

Stop generating old incoming.debian.org

Signed-off-by: Mark Hymers <mhy@debian.org>

Don't use privacy temporarily to work out which buildds need reconfiguring

Signed-off-by: Mark Hymers <mhy@debian.org>

Don't break on too many links

Signed-off-by: Mark Hymers <mhy@debian.org>

Remove old variables

Signed-off-by: Mark Hymers <mhy@debian.org>

Use chronic

Signed-off-by: Mark Hymers <mhy@debian.org>

Drop created and modified columns from bin_contents and src_contents

Debian_arch_to_Debian_triplet: handle linux-${arch}

Fix DirectoryMatch

Signed-off-by: Mark Hymers <mhy@debian.org>

We don't want emails every 15 minutes

Signed-off-by: Mark Hymers <mhy@debian.org>

Don't sync incoming - there's no point

Signed-off-by: Mark Hymers <mhy@debian.org>

Use the right option

Signed-off-by: Mark Hymers <mhy@debian.org>

Change how we generate incoming.debian.org

Signed-off-by: Mark Hymers <mhy@debian.org>

Move canonical incoming.debian.org location

Signed-off-by: Mark Hymers <mhy@debian.org>

export-suite: Add --relative option.

apache.conf-incoming: Grant access to home.mraw.org

process-new: Indicate which packages were not include in upload

Show "NOT UPLOADED" besides overrides for packages that were not
included in the upload.

Only inject processed uploads into signature_history

Uploads that were 'skipped' should not be recorded in
signature_history as they will be repeocessed later.

daklib/regexes.py: Do not allow colons in filenames.

Colons should not be used anywhere as the epoch part of versions is
not included in the filename.

daklib/checks.py: Add instructions how to get an upload processed again.

daklib/gpg.py: Ignore SIGEXPIRED and KEYEXPIRED

apache.conf-incoming: Grant access to mahler.debian.net

Bug: https://bugs.debian.org/754742

daklib/gpg.py: Ignore NOTATION_{NAME,DATA} keywords.

show-deferred: Do not require a valid signature.

daklib/gpg.py: Also handle SIGEXPIRED keyword.

daklib/gpg.py: Handle a few more keywords for invalid or and expired signatures.

Drop unique .changes name requirement and allow ftpteam to forget seen files.

dak admin: add forget-signature subcommand

The forget-signature command removes the entry for a given file from
the signature_history. This allows reprocessing the file.

Add replay protection for uploaded changes files.

Look in signature_history table if the changes was already seen to
prevent re-uploading old changes files.

daklib/gpg.py: Use timestamp instead of just date.

daklib/gpg.py: Raise an error if we see unexpected output from gpg.

Fix breakage on missing w-b dump for an arch

The w-b admins may take some time until they have a new architecture all
set up. And until they do generate the buildd.d.o/stats/$arch-all.txt we shouldn't
break on it missing.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Add arm64 keyring to queued configuration

Signed-off-by: Mark Hymers <mhy@debian.org>

Add arm64

Signed-off-by: Mark Hymers <mhy@debian.org>

built_in_suite: Do not consider architecture 'source'

daklib.architecture.match_architecture would consider 'source' as an
invalid architecture name.

match_architecture: Fix raising exception if an architecture is unknown

process-new: Make again use of Package-List field.

daklib/dbconn.py: Add MetadataProxy that behaves like a dict

scripts/debian/moveftp.sh: use mv --no-clobber

config/debian/dak.conf: Allow some source-only uploads.

PackageList: use correct values for section and component

Add more fine-grained options for source-only uploads.

Use Package-List to look for NEW binaries.

daklib.upload.Source: expose Package-List.

daklib.upload.Binary: add name property.

PackageListEntry: Rename package_type to type.

PackageListEntry should be similar to daklib.upload.Binary.

The Binary field of source packages is comma-seperated.

Initial pass at dak web server

Signed-off-by: Mark Hymers <mhy@debian.org>

daklib/packagelist.py: Additional sanity checks.

extract_component_from_section: avoid unneeded database lookup

Add parser for Package-List field.