buildpackage: add sloppy mode to build upstream tarballs

When starting with Debian _and_ git people often stumble over a mismatch
between what dpkg-source expects to be in the tarball and the generated
tarball for various reasons. Give them a way to say:
   "Use what I have on my current debian branch"
as upstream source.

Closes: #861004

4 files changed, 92 insertions, 10 deletions

diff --git a/docs/chapters/special.sgml b/docs/chapters/special.sgml
index d7257b53..5000c1a7 100644
--- a/docs/chapters/special.sgml
+++ b/docs/chapters/special.sgml
@@ -170,5 +170,63 @@ EOF
    The import works incrementally; you can import new versions on top of
    already imported ones for e.g. easy review of changes.
     </para>
+
     </sect1>
+    <sect1 id="gbp.special.sloppytarball">
+    <title>Sloppy tarball creation</title>
+    <para>
+      By default &gbp-buildpackage; uses <filename>debian/changelog</filename> to detect
+      the upstream version and build the corrsponding tarball either via &pristine-tar; or
+      by using <command>git archive</command> directly. This ensures that the tarball matches
+      what's in the Debian archive already.
+    </para>
+    <para>
+      But there might be cases where you don't want to follow these
+      packaging practice and create a fresh <emphasis>dirty</emphasis>
+      tarball that contains additional changes. Such tarballs are not
+      suitable for upload into the archive but might be helpful in local testing.
+    </para>
+    <para>
+      Especially when getting started and when you first want to get a
+      package built and dive into &gbp-dch;, &git-pbuilder; and the 3.0
+      (quilt) format later to find out what exactly
+<programlisting>
+dpkg-source: info: local changes detected, the modified files are:
+ hello-debhelper/configure
+dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/hello-debhelper_2.8-1.diff.har2Xx
+dpkg-source: info: you can integrate the local changes with dpkg-source --commit
+</programlisting>
+means.
+    </para>
+    <para>
+      For that you can force &gbp-buildpackage; to create a tarball
+      from the <emphasis>debian-branch</emphasis> dropping
+      the <filename>debian/</filename> directory. This will give you a
+      tarball that is very close to your current working copy except
+      for the packaging.
+    </para>
+<programlisting>
+  &gbp-buildpackage --git-ignore-new --git-force-create --git-upstream-tree=SLOPPY --git-no-pristine-tar
+</programlisting>
+    <para>
+      The important part is the <emphasis>SLOPPY</emphasis> above. The
+      other options are there to prevent &gbp-buildpackage; from using
+      &pristine-tar;, to force the creation of a new tarball and to
+      not abort if you have changes in your current working copy.
+
+      If you're still getting an error about <emphasis>unexpected
+      upstream changes</emphasis> make sure you have no uncomitted
+      changes in you workig copy (changes in
+      the <filename>debian/</filename> are o.k.) and no files that are
+      ignored by git
+      via <filename>.gitignore</filename>. A <command>git clean
+      -dfx</command> can help here.
+    </para>
+    <para>
+      Please don't use this tarball to upload to the Debian archive
+      since changes outside <filename>debian/</filename> need to be
+      represented by patches in <filename>debian/patches</filename>.
+      See the <command>dpkg-source</command> manpage for
+      details.
+   </para>
 </chapter>
diff --git a/docs/manpages/gbp-buildpackage.sgml b/docs/manpages/gbp-buildpackage.sgml
index 2faced25..63b285ef 100644
--- a/docs/manpages/gbp-buildpackage.sgml
+++ b/docs/manpages/gbp-buildpackage.sgml
@@ -537,18 +537,18 @@
           <replaceable>INDEX</replaceable> exports the current index whereas
           the special name <replaceable>WC</replaceable> exports the current
           working copy as is.
-	  </para>
-	  <para>
-	    Note that using <replaceable>WC</replaceable> enables the
-	    <option>--git-ignore-branch</option>
-	    and <option>--git-ignore-new</option> options as well
-	    since when exporting the working copy there's no point in
-	    enforcing any branch or modification checks.
+          </para>
+          <para>
+            Note that using <replaceable>WC</replaceable> enables the
+            <option>--git-ignore-branch</option>
+            and <option>--git-ignore-new</option> options as well
+            since when exporting the working copy there's no point in
+            enforcing any branch or modification checks.
           </para>
         </listitem>
       </varlistentry>
       <varlistentry>
-        <term><option>--git-upstream-tree=</option><replaceable>[TAG|BRANCH|TREEISH]</replaceable>
+        <term><option>--git-upstream-tree=</option><replaceable>[BRANCH|SLOPPY|TAG|TREEISH]</replaceable>
         </term>
         <listitem>
           <para>
@@ -556,8 +556,13 @@
           <replaceable>TAG</replaceable> (the default) looks at a tag corresponding to the
           version in the changelog. <replaceable>BRANCH</replaceable> looks at
           the upstream branch given via the
-          <option>--git-upstream-branch</option> option. Other values are
-          interpreted as treeishs.
+          <option>--git-upstream-branch</option> option. The <replaceable>SLOPPY</replaceable>
+	  option looks at the debian branch given via
+	  the <option>--git-debian-branch</option> and drops
+	  the <filename>debian/</filename> dir.
+	  </para>
+	  <para>
+	  Other values are interpreted as treeishs.
           </para>
           <para>
           This doesn't have any effect if <option>--git-pristine-tar</option>
diff --git a/gbp/scripts/buildpackage.py b/gbp/scripts/buildpackage.py
index 6cac603f..0922ff9f 100755
--- a/gbp/scripts/buildpackage.py
+++ b/gbp/scripts/buildpackage.py
@@ -336,6 +336,9 @@ def get_upstream_tree(repo, source, options):
         if not repo.has_branch(options.upstream_branch):
             raise GbpError("%s is not a valid branch" % options.upstream_branch)
         upstream_tree = options.upstream_branch
+    elif options.upstream_tree.upper() == 'SLOPPY':
+        tree_name = "%s^{tree}" % options.debian_branch
+        upstream_tree = repo.tree_drop_dirs(tree_name, ["debian"])
     else:
         upstream_tree = options.upstream_tree
     if not repo.has_treeish(upstream_tree):
diff --git a/tests/component/deb/test_buildpackage.py b/tests/component/deb/test_buildpackage.py
index f8294115..b1499361 100644
--- a/tests/component/deb/test_buildpackage.py
+++ b/tests/component/deb/test_buildpackage.py
@@ -19,6 +19,7 @@
 import hashlib
 import os
 import subprocess
+import tarfile
 
 from tests.component import (ComponentTestBase,
                              ComponentTestGitRepository)
@@ -141,6 +142,21 @@ class TestBuildpackage(ComponentTestBase):
             self.assertTrue(os.path.exists(t), "Tarball %s not found" % t)
 
     @RepoFixtures.quilt30()
+    def test_sloppy_tarball_generation(self, repo):
+        """Test that generating tarball from Debian branch works"""
+        tarball = '../hello-debhelper_2.8.orig.tar.gz'
+        self.add_file(repo, 'foo.txt')
+        self._test_buildpackage(repo, ['--git-force-create',
+                                       '--git-upstream-tree=SLOPPY'])
+        self.assertTrue(os.path.exists(tarball))
+        t = tarfile.open(name=tarball, mode="r:gz")
+        names = t.getnames()
+        for f in ['hello-debhelper-2.8/build-aux',
+                  'hello-debhelper-2.8/foo.txt']:
+            self.assertIn(f, names)
+        self.assertNotIn('hello-debhelper-2.8/debian', names)
+
+    @RepoFixtures.quilt30()
     def test_export_dir_buildpackage(self, repo):
         """Test that building with a export dir works"""
         self._test_buildpackage(repo, ['--git-export-dir=../foo/bar'])